Re: [xml] Lower speed with greater xmlParseChunk() chunks?

Date view Thread view Subject view Author view

From: Daniel Veillard (Daniel.Veillard@w3.org)
Date: Sun Oct 15 2000 - 15:35:04 EDT

On Sun, Oct 15, 2000 at 12:14:42PM -0700, Wayne Davison wrote:
>
> On Sun, 15 Oct 2000 rolf@pointsman.de wrote:
> > Playing around I found one very strange behavior. Parsing speed slows
> > (dramatically) down, if the chunks of data are big.
>
> I noticed something similar, but I never tried to quantify it. I believe
> that the problem is that the code likes to keep shifting the buffer as it
> parses it. A while back I removed one such "SHRINK" call (only when the
> data was being pushed), but I seem to recall that it didn't eliminate the
> shifting. I decided to deal with this by just calling the push function
> in a loop with a guaranteed small chunk size and then forgot to bring up
> the subject.

   yeah that sounds like something similar ... I should modify
the SHRINK macro to activate the shift only if there isn't much data
left in the buffer. This may lead to a serious improvement ...
yes I think I should try this before 2.2.5 release because it sounds
harmless and a possible serious improvement !

> Another problem with the code is that it copies all the data that you push
> into an input buffer, and then translates all that data into another UTF8
> buffer, so you can end up consuming another 2x the size of your push
> buffer in memory.

   yes, I could restrain it at that level too ...

> I've thought that the push routine should be modified to only put a small
> chunk of memory at a time into the input buffer, effectively moving my
> buffer-dividing loop into the xml internals.

  yeah adding a small loop in xmlParseChunk() is another option too
a blocking factor of 4k sounds a good default a priori ...

Daniel 

-- 
Daniel.Veillard@w3.org | W3C, INRIA Rhone-Alpes  | Today's Bookmarks :
Tel : +33 476 615 257  | 655, avenue de l'Europe | Linux XML libxml WWW
Fax : +33 476 615 207  | 38330 Montbonnot FRANCE | Gnome rpm2html rpmfind
 http://www.w3.org/People/all#veillard%40w3.org  | RPM badminton Kaffe
----
Message from the list xml@rpmfind.net
Archived at : http://xmlsoft.org/messages/
to unsubscribe: echo "unsubscribe xml" | mail  majordomo@rpmfind.net

Date view Thread view Subject view Author view

This archive was generated by hypermail 2b29 : Sun Oct 15 2000 - 15:43:22 EDT