[reportlab-users] ValueError: no memory after 4GB

[Robin Becker]

Hi Theodore,

sorry you are having a problem.

At the very least I expect the BMP to need  100000 x 100000 x 3 bytes which is at least 30Gb; since the image is 
converted by PIL it may well require twice that if things go well.

However, looking in the code that message is output by the initial allocation of the in memory image buffer for the 
gstate object) which is trying to allocate w * h * nchan bytes (ie 30Gb) using this code

p->buf = PyMem_Malloc(n=w*h*nchan);

so the python memory manager returns NULL and the gstate creation fails and returns NULL to python which handles the 
error. Given that python is telling you that it cannot allocate the memory probably means that your process is either 
constrained in some way or that this additional 30Gb is too much.

If you are using a really modern version of reportlab you can try to use the other available backend which we built 
recently  rlPyCairo (needs version >=3.5.61). You can pip install rlPyCairo and follow the instructions see 
https://pypi.org/project/rlPyCairo/. I doubt that this will actually improve things as you will have even more memory 
used by additional python code and modules.

To assist you in your debugging it might be useful to use methods from this page

https://stackoverflow.com/questions/938733/total-memory-used-by-python-process

although even you should be able to allocate the required memory it may not be possible if the memory has become 
fragmented ie the 30Gb may not be contiguous.

In addition PIL has limits on the image size and will issue warnings and/or fail when trying to deal with such images.

You could try simple experiments in your environment ie

$ python -c"from reportlab.graphics._renderPM import gstate;gs=gstate(10000,10000)"
$ python -c"from reportlab.graphics._renderPM import gstate;gs=gstate(100000,100000)"

on my machine the first is OK, but the second fails, but I only have 16Gb memory. I suppose the above is the minimal 
case ie no extra memory except that used by the import and extension.


-- 
Robin Becker

On 06/04/2021 20:04, Shaw, Theodore wrote:
> Hi,
> 
> I'm trying to convert an SVG to a BMP that's about 100,000 pixels by 100,000 pixels. My code looks like
> 
> drawing = svg2rlg( 'infile.svg')
> renderPM.drawToFile(drawing, 'outfile.bmp', fmt='BMP')
> 
> When I run it I get "ValueError: _renderPM.gstate: no memory."
............
> 
> Is this expected? Is there any workaround?
> 
> Thank you very much.
> 
> _______________________________________________
> reportlab-users mailing list
> reportlab-users at lists2.reportlab.com
> https://pairlist2.pair.net/mailman/listinfo/reportlab-users