[reportlab-users] Intermittent failure to find font error

[Alex Buck]

Hi Everyone,

I had been experiencing an issue on our production webservers as Ian
who posted last year about the same problem [1]. After a lot of
research and investigation I believe I have come up with a reasonable
explanation to thisissue. First a little bit about the production
environment. We are currently serving reports to our users via django
through mod_wsgi running apache. Our configuration allows for 20
concurrent django processes with a single thread running in each
process. I was able to reproduce this problem in a development
environment by doing the following.

1. restart apache
2. send many requests (httperf was tool of choice) to apache/django to
ensure all processes have been invoked and used at least once. The requests
I was sending had nothing to do with reportlab so it would not be loaded
until I went to run the report.
3. proceed to download the report where intermittently it would fail with
same DarkGarden.afm error.

Now if I reverse steps 2 and 3 and download the report right after an
apacherefresh the problem will not occur on subsequent re-entries to
download thereport.

What I assume to be happening is when Apache restarts and receives a
report view request initially it will load everything into into
primary python sub
interpreter. (Another assumption being that secondary interpreter
process memory is copied from the primary sub interpreter). Apache
will create a sub
interpreter for every process we have [2]. Furthermore, each
interpreter should live the whole life the process [3]. There is a
library deep within
reportlab which calls external C code (_rl_accel.c) to try and
accelerate functions like finding fonts (*relevant C code). These
functions cache a
reference to the loaded font cache in the python interpreter process.
If the cache pointer is assigned after a bunch of initial requests it
will be
assigned with a reference from a secondary interpreter instead of the
primary interpreter.  Therefore additional requests could mix code and
data
from different interpreters resulting in the IOError "not accessible
in restricted mode". [4]

The quick fix is to disable the C extensions within reportlab by
moving the shared object (_rl_accel.so). Reportlab will degrade
gracefully and use
python code in place of these acceleration extensions. This ensures no
python level references are stored in external C memory and shared
between
interpreters. After the following there is no further IOError
exceptions being raised.

I have included the snippet of the code which I believe is the cause
of the error. Could the reportlab developers please look into this
issue and fix
the acceleration libraries so as to not share python objects from
different interpreters?

Thanks for all the great work guys :)

Alex

*reference
-------------------------------------------------------------------------------------------

[1] http://two.pairlist.net/pipermail/reportlab-users/2009-January/007928.html
[2] http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Application_Global_Variables
[3] http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading#Python_Sub_Interpreters
[4] http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Multiple_Python_Sub_Interpreters

*relevant C code
-------------------------------------------------------------------------------------------

src/rl_addons/rl_accel/_rl_accel.c:1035

[1]   static PyObject *_pdfmetrics_fonts = NULL;  /*the fontName to font map
from pdfmetrics*/
       static PyObject *_pdfmetrics_ffar = NULL;   /**findFontAndRegister
from pdfmetrics**/
       static PyObject *getFontU(PyObject *module, PyObject *args, PyObject
*kwds)
       {
           PyObject *fontName=NULL, *_o1=NULL, *_o2=NULL, *res=NULL;
           static char *argnames[] = {"fontName",NULL};
           if(!PyArg_ParseTupleAndKeywords(args, kwds, "O", argnames,
&fontName)) return NULL;
   [2]       if(!_pdfmetrics_fonts){
               res = PyImport_ImportModule("reportlab.pdfbase.pdfmetrics");
               if(!res) ERROR_EXIT();
   [3]        _o1 = _GetAttrString(res,"_fonts");
               if(!_o1) ERROR_EXIT();
               _o2 = _GetAttrString(res,"findFontAndRegister");
               if(!_o2) ERROR_EXIT();
               _pdfmetrics_fonts = _o1;
               _pdfmetrics_ffar = _o2;
               Py_DECREF(res); _o1 = _o2 = res = NULL;
               }
     [4]  if((res = PyObject_GetItem(_pdfmetrics_fonts,fontName))) return
res;
           if(!PyErr_ExceptionMatches(PyExc_KeyError)) ERROR_EXIT();
           ...
     [5]  res = PyObject_CallObject(_pdfmetrics_ffar,_o1);
           Py_DECREF(_o1); /**NB this should decremnent fontName as well**/
           return res;
           ....

[1] Declares global pointer to _fonts cache
[2] If that pointer is NULL load the pdfmetrics namespace
[3] Get the font cache from pdfmetrics from the current interpreter
[4] Check to see if pdf metrics exists in cache and return result. However
pdfmetrics could belong to any sub-interpreter.
[5] If [4] fails then attempt the bruteforce search where IOError happens
passing in _fonts cache from another sub-interpreter.