is this better to be an enum rather than concrete type or boxed dyn?
I thought buffer protocol help to avoid this situation.

I could probably add a Dyn variant that's just Box<dyn Deref> if eventually needed, but as is these are the most common forms of borrowing, and they all just have the underlying borrow stored as a reference/pointer, so no function calls are necessary to Deref it (unlike a dyn Deref)

Maybe I don't know rust pratice well, so this is trying discussion.
When I am seeing BorrowValue for specific type like PyBytesLike, it doesn't use all the enum fields but its borrowed type is BorrowedValue. I think this is giving unnessassary cost.
using enum for each case is ok - because it is what it need to be. But BorrowedValue itself as enum is hiding somewhat not expected to be hided.
my suggestion is, defining specific type for each borrowed types. BorrowedBytesLike or BorrowedRwBytes. And BorrowedValue itself as a trait (if it is needed). We can find better abstraction or we can use dyn BorrowedValue in future complicated scenarios.

I think this is good because with the buffer protocol PyBytesLike and PyRwBytesLike will become extensible and having a flexible return type from borrow_value will make it easier for implementer's of the buffer protocol. I don't this will actually have much cost as all variants probably will optimize down to a simply deref anyways.