我正在使用熊猫做环形缓冲区,但内存使用量不断增长.我究竟做错了什么?
这是代码(从问题的第一篇文章稍微编辑):
import pandas as pd import numpy as np import resource tempdata = np.zeros((10000,3)) tdf = pd.DataFrame(data=tempdata, columns = ['a', 'b', 'c']) i = 0 while True: i += 1 littledf = pd.DataFrame(np.random.rand(1000, 3), columns = ['a', 'b', 'c']) tdf = pd.concat([tdf[1000:], littledf], ignore_index = True) del littledf currentmemory = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss if i% 1000 == 0: print 'total memory:%d kb' % (int(currentmemory)/1000)
这就是我得到的:
total memory:37945 kb total memory:38137 kb total memory:38137 kb total memory:38768 kb total memory:38768 kb total memory:38776 kb total memory:38834 kb total memory:38838 kb total memory:38838 kb total memory:38850 kb total memory:38854 kb total memory:38871 kb total memory:38871 kb total memory:38973 kb total memory:38977 kb total memory:38989 kb total memory:38989 kb total memory:38989 kb total memory:39399 kb total memory:39497 kb total memory:39587 kb total memory:39587 kb total memory:39591 kb total memory:39604 kb total memory:39604 kb total memory:39608 kb total memory:39608 kb total memory:39608 kb total memory:39608 kb total memory:39608 kb total memory:39608 kb total memory:39612 kb
不确定它是否与此相关:
https://github.com/pydata/pandas/issues/2659
使用Anaconda Python在MacBook Air上测试