[reportlab-users] too many open files with inline images

[Robin Becker]

Volker Haas wrote:

> I get an error (too many open files) when using lots of inline images 

> inside paragraphs.

> 

> Sample code:

> 

> from reportlab.platypus.paragraph import Paragraph

> from reportlab.platypus.doctemplate import SimpleDocTemplate

> from reportlab.lib.styles import ParagraphStyle

> 

> import os

> doc = SimpleDocTemplate('test.pdf')

> elements = []

> 

> for count in range(1024):

>    print len(os.listdir("/proc/%d/fd" % os.getpid()))

>    p = Paragraph('text <img src="math2.png" width="2cm" height="0.5cm" 

> />' , ParagraphStyle('Normal'))

>    elements.append(p)

> doc.build(elements)

> 

> OUTPUT:

> 

> Traceback (most recent call last):

>  File "test_fd.py", line 13, in <module>

> OSError: [Errno 24] Too many open files: '/proc/32336/fd'

> 


I'm pretty sure this bug is not going to go away unless we rewrite the 
ImageReader fp handling stuff. The intent is that ImageReader should be able to 
handle a python file handle, a PIL image or a simple filename. To avoid 
excessive work we open the image at the front rather than at the back end. I 
suspect the only fix here is to delay opening(or reopen) in the case when when a 
  path is given. In the cases when either a PIL image or an open file is passed 
I don't think there's any obvious way to cut down on the number of open files 
other than converting them into memory based objects.

Of course it could be that we are being a bit stupid in not re-using the images 
if they're repeated often or that we are holding on to the image objects too long.


> If the print statement is omitted (which isn't really platform 

> independent ;)) I get an IOError (IOError: Cannot open resource 

> "math.png" fileName=math.png) . But I am pretty sure this only happens 

> because no new file handle could be aquired. The sample code works if 

> less images are used (my machines limit for open file handles is 1024 

> per process).

> I am not attaching the sample image, b/c the sample code behaves in the 

> same way when the image file is not existent.

> 

> Regards,

> Volker

> 



-- 
Robin Becker