作者:w康d | 来源:互联网 | 2023-05-24 20:23
我在Cython中编写了一个Python 2.7扩展模块. 如何创建一个实现新式缓冲区接口的Python对象,该接口包含C库提供给我的大块内存? 内存块只是一串字节,而不是结构或多维数组.我给了一个const void *
指针和一个长度,以及一些指针保持有效的细节.
我无法复制内存 - 这会破坏我的应用程序的性能.
使用我可以简单使用的旧式缓冲区对象PyBuffer_FromMemory()
,但我似乎无法找到一种类似的简单方法来生成新式缓冲区对象.
我是否必须创建自己的实现缓冲区接口的类?或者Cython提供了一种简单的方法吗?
我已经阅读了Cython文档中的Unicode和Passing Strings和Typed Memoryviews页面,但文档不精确且不够完整,并且没有类似于我想要做的示例.
这是我尝试过的(test.pyx
):
from libc.stdlib cimport malloc
from libc.string cimport memcpy
## pretend that this function is in some C library and that it does
## something interesting. (this function is unrelated to the problem
## I'm experiencing -- this is just an example function that returns a
## chunk of memory that I want to wrap in an object that follows the
## new buffer protocol.)
cdef void dummy_function(const void **p, size_t *l):
cdef void *tmp = malloc(17)
memcpy(tmp, "some test\0 bytes", 17)
p[0] = tmp
l[0] = 17
cpdef getbuf():
cdef const void *cstr
cdef size_t l
dummy_function(&cstr, &l)
## error: test.pyx:21:20: Invalid base type for memoryview slice: void
#cdef const void[:] ret = cstr[:l]
## error: test.pyx:24:9: Assignment to const 'ret'
#cdef const char[:] ret = cstr[:l]
## error: test.pyx:27:27: Cannot convert 'void const *' to memoryviewslice
#cdef char[:] ret = cstr[:l]
## this next attempt cythonizes, but raises an exception:
## $ python -c 'import test; test.getbuf()'
## Traceback (most recent call last):
## File "", line 1, in
## File "test.pyx", line 15, in test.getbuf (test.c:1411)
## File "test.pyx", line 38, in test.getbuf (test.c:1350)
## File "stringsource", line 614, in View.MemoryView.memoryview_cwrapper (test.c:6763)
## File "stringsource", line 321, in View.MemoryView.memoryview.__cinit__ (test.c:3309)
## BufferError: Object is not writable.
cdef char[:] ret = (cstr)[:l]
## this raises the same exception as above
#cdef char[:] ret = (cstr)[:l]
return ret
Richard Hans..
5
您可以通过定义和特殊方法来定义实现缓冲区协议的扩展类型.例如:__getbuffer__
__releasebuffer__
from cpython.buffer cimport PyBuffer_FillInfo
from libc.stdlib cimport free, malloc
from libc.string cimport memcpy
cdef void dummy_function(const void **p, size_t *l):
cdef void *tmp = malloc(17)
memcpy(tmp, "some test\0 bytes", 17)
p[0] = tmp
l[0] = 17
cdef void free_dummy_data(const void *p, size_t l, void *arg):
free(p)
cpdef getbuf():
cdef const void *p
cdef size_t l
dummy_function(&p, &l)
return MemBuf_init(p, l, &free_dummy_data, NULL)
ctypedef void dealloc_callback(const void *p, size_t l, void *arg)
cdef class MemBuf:
cdef const void *p
cdef size_t l
cdef dealloc_callback *dealloc_cb_p
cdef void *dealloc_cb_arg
def __getbuffer__(self, Py_buffer *view, int flags):
PyBuffer_FillInfo(view, self, self.p, self.l, 1, flags)
def __releasebuffer__(self, Py_buffer *view):
pass
def __dealloc__(self):
if self.dealloc_cb_p != NULL:
self.dealloc_cb_p(self.p, self.l, self.dealloc_cb_arg)
# Call this instead of constructing a MemBuf directly. The __cinit__
# and __init__ methods can only take Python objects, so the real
# constructor is here. See:
# https://mail.python.org/pipermail/cython-devel/2012-June/002734.html
cdef MemBuf MemBuf_init(const void *p, size_t l,
dealloc_callback *dealloc_cb_p,
void *dealloc_cb_arg):
cdef MemBuf ret = MemBuf()
ret.p = p
ret.l = l
ret.dealloc_cb_p = dealloc_cb_p
ret.dealloc_cb_arg = dealloc_cb_arg
return ret
使用上面的(命名test.pyx
),您将获得以下行为:
$ python -c 'import test; print repr(memoryview(test.getbuf()).tobytes())'
'some test\x00 bytes\x00'
我不知道是否有更简单的方法.
1> Richard Hans..:
您可以通过定义和特殊方法来定义实现缓冲区协议的扩展类型.例如:__getbuffer__
__releasebuffer__
from cpython.buffer cimport PyBuffer_FillInfo
from libc.stdlib cimport free, malloc
from libc.string cimport memcpy
cdef void dummy_function(const void **p, size_t *l):
cdef void *tmp = malloc(17)
memcpy(tmp, "some test\0 bytes", 17)
p[0] = tmp
l[0] = 17
cdef void free_dummy_data(const void *p, size_t l, void *arg):
free(p)
cpdef getbuf():
cdef const void *p
cdef size_t l
dummy_function(&p, &l)
return MemBuf_init(p, l, &free_dummy_data, NULL)
ctypedef void dealloc_callback(const void *p, size_t l, void *arg)
cdef class MemBuf:
cdef const void *p
cdef size_t l
cdef dealloc_callback *dealloc_cb_p
cdef void *dealloc_cb_arg
def __getbuffer__(self, Py_buffer *view, int flags):
PyBuffer_FillInfo(view, self, self.p, self.l, 1, flags)
def __releasebuffer__(self, Py_buffer *view):
pass
def __dealloc__(self):
if self.dealloc_cb_p != NULL:
self.dealloc_cb_p(self.p, self.l, self.dealloc_cb_arg)
# Call this instead of constructing a MemBuf directly. The __cinit__
# and __init__ methods can only take Python objects, so the real
# constructor is here. See:
# https://mail.python.org/pipermail/cython-devel/2012-June/002734.html
cdef MemBuf MemBuf_init(const void *p, size_t l,
dealloc_callback *dealloc_cb_p,
void *dealloc_cb_arg):
cdef MemBuf ret = MemBuf()
ret.p = p
ret.l = l
ret.dealloc_cb_p = dealloc_cb_p
ret.dealloc_cb_arg = dealloc_cb_arg
return ret
使用上面的(命名test.pyx
),您将获得以下行为:
$ python -c 'import test; print repr(memoryview(test.getbuf()).tobytes())'
'some test\x00 bytes\x00'
我不知道是否有更简单的方法.