[reportlab-users] Performance problem for lineplots with many values (with possible solution)

[Robin Becker]

Sebastian,

I looked at your proposed improvement and thought this must be obviously a 
O(n)running time versus O(n**2) algorithm problem. However,

when I run the following code

#########################################
import sys, time
def main(n):
	print 20*'#','START n=%s'%n,20*'#'
	row = [(i,i+1) for i in xrange(2*n)]
	print 'existing algorithm',
	t0 = time.time()
	points = []
	for xy in row:
		points += [xy[0],xy[1]]
	t1 = time.time()
	print 'took %.2f seconds' % (t1-t0)

	print 'existing algorithm using list',
	t0 = time.time()
	points = []
	for xy in row:
		points += list(xy[:2])
	t1 = time.time()
	print 'took %.2f seconds' % (t1-t0)

	print 'existing algorithm using list assuming length 2',
	t0 = time.time()
	points = []
	for xy in row:
		points += list(xy)
	t1 = time.time()
	print 'took %.2f seconds' % (t1-t0)

	print 'map(list,row)',
	t0 = time.time()
	points = map(list,row)
	t1 = time.time()
	print 'took %.2f seconds' % (t1-t0)

	print '[list(xy) for xy in row]',
	t0 = time.time()
	points = [list(xy) for xy in row]
	t1 = time.time()
	print 'took %.2f seconds' % (t1-t0)

	print '[[xy[0],xy[1]] for xy in row]',
	t0 = time.time()
	points = [[xy[0],xy[1]] for xy in row]
	t1 = time.time()
	print 'took %.2f seconds' % (t1-t0)

	print 'append algorithm',
	t0 = time.time()
	points = [].append
	for xy in row:
		points([xy[0],xy[1]])
	points = points.__self__
	t1 = time.time()
	print 'took %.2f seconds' % (t1-t0)

	print 20*'#','END	n=%s'%n,20*'#','\n\n'

if __name__=='__main__':
	if len(sys.argv)==1:
		N = [86000]
	else:
		N = map(int,sys.argv[1:])
	for n in N:
		main(n)
#########################################

I see this

C:\code\tests>tpoints.py 86000 860000
#################### START n=86000 ####################
existing algorithm took 0.08 seconds
existing algorithm using list took 0.12 seconds
existing algorithm using list assuming length 2 took 0.12 seconds
map(list,row) took 0.16 seconds
[list(xy) for xy in row] took 0.28 seconds
[[xy[0],xy[1]] for xy in row] took 0.22 seconds
append algorithm took 0.19 seconds
#################### END   n=86000 ####################


#################### START n=860000 ####################
existing algorithm took 0.84 seconds
existing algorithm using list took 1.33 seconds
existing algorithm using list assuming length 2 took 1.25 seconds
map(list,row) took 3.41 seconds
[list(xy) for xy in row] took 2.97 seconds
[[xy[0],xy[1]] for xy in row] took 2.70 seconds
append algorithm took 2.47 seconds
#################### END   n=860000 ####################

so the existing algorithm appears to win. Can you try this script on your 
machine? What python are you using etc etc?

Perhaps there's some other reason why your script has a long running time.
--
Robin Becker

On 30/01/2013 10:17, Sebastian Messing wrote:

> Hi!

>

> I tried to generate a lineplot with joinedLines=1. The lineplot has three lines,

> each line with 86000 values. The generation of the lineplot is very, very slow

> (hours).

>

> I use ReportLab 2.6 but I also found the problem in ReportLab daily from

> 01/29/2013 in /src/reportlab/graphics/charts/lineplots.py:

> 276  # Iterate over data columns.

> 277  if self.joinedLines:

> 278      points = []

> 279      for xy in row:

> 280          points += [xy[0], xy[1]]

>

> If I use a list comprehension instead, the plot is generated within seconds or

> minutes:

> 278  points = [[xy[0], xy[1]] for xy in row]

> 279

> 280

>

> Also linecharts.py contains the same performance problem but I don't use that.

>

> Best regards

> Sebastian Messing