Skip to content

packer: add buf_size option #604

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
methane merged 5 commits into from
May 5, 2024
+15 −27
Merged

packer: add buf_size option #604

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
edb61db
Allow users to specify Packer internal buf_size
rtobar Jul 18, 2023
db39717
Merge branch 'main' into bufsize-argument
methane May 5, 2024
fdf5c1c
change default buffer size to 256KiB
methane May 5, 2024
fe30d22
Merge branch 'main' into bufsize-argument
methane May 5, 2024
09825cb
update fallback
methane May 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
update fallback
  • Loading branch information
@methane
methane committed May 5, 2024
commit 09825cb5290e808f39f29f913f3be8b9c39421b7
2 changes: 1 addition & 1 deletion msgpack/_packer.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ cdef inline int PyBytesLike_CheckExact(object o):
return PyBytes_CheckExact(o) or PyByteArray_CheckExact(o)


cdef class Packer(object):
cdef class Packer:
"""
MessagePack Packer

Expand Down
2 changes: 1 addition & 1 deletion msgpack/_unpacker.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ def unpackb(object packed, *, object object_hook=None, object list_hook=None,
raise ValueError("Unpack failed: error = %d" % (ret,))


cdef class Unpacker(object):
cdef class Unpacker:
"""Streaming unpacker.

Arguments:
Expand Down
27 changes: 5 additions & 22 deletions msgpack/fallback.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,7 @@ class Unpacker:
def __init__(
self,
file_like=None,
*,
read_size=0,
use_list=True,
raw=False,
Expand Down Expand Up @@ -650,39 +651,21 @@ class Packer:
The error handler for encoding unicode. (default: 'strict')
DO NOT USE THIS!! This option is kept for very specific usage.

Example of streaming deserialize from file-like object::

unpacker = Unpacker(file_like)
for o in unpacker:
process(o)

Example of streaming deserialize from socket::

unpacker = Unpacker()
while True:
buf = sock.recv(1024**2)
if not buf:
break
unpacker.feed(buf)
for o in unpacker:
process(o)

Raises ``ExtraData`` when *packed* contains extra bytes.
Raises ``OutOfData`` when *packed* is incomplete.
Raises ``FormatError`` when *packed* is not valid msgpack.
Raises ``StackError`` when *packed* contains too nested.
Other exceptions can be raised during unpacking.
:param int buf_size:
Internal buffer size. This option is used only for C implementation.
"""

def __init__(
self,
*,
default=None,
use_single_float=False,
autoreset=True,
use_bin_type=True,
strict_types=False,
datetime=False,
unicode_errors=None,
buf_size=None,
):
self._strict_types = strict_types
self._use_float = use_single_float
Expand Down