I've got a pretty big xml file which is loaded with umbraco.library:GetXmlDocumentByUrl. I use xpath to get an element which is pretty deep in the tree. Here is my xpath:
This seems pretty slow. Is there a way to optimize my code? Can the xpath be faster some other way or can I somehow fragment the xml file more for better performance?
That xpath looks good in that you aren't using '//' or 'descendant-or-self' but are instead specifying exactly how to get to the subitem you want.
It may be that you have a LOT of subitem entries in the XML?
But more likely, the GetXmlDocumentByUrl() is sluggish retrieving a large XML file due to latency on the http request and bandwidth to return the xml itself. If you can, use the optional timeout parameter with GetXmlDocumentByUrl() to reduce the frequency of the request for the xml file itself. And macro caching will also be your friend. You might also want to look at Darren's FeedCache package, which is excellent.
I've been debugging the code and getting the xml with GetXmlDocumentByUrl works fine. The slow part is where I do my xpath. There are a lot of subitem entries indeed. I was hoping that I could improve my xpath somehow. Maybe narrow down in which element I need to search. Thanks for helping.
If each step has a lot of items it might help to hand-pick the one you're after (if at all possible) e.g.
$index/index[1]/chambers[1]/etc...
- though that's probably not possible for you.
You should try using a key - tricky part is it only works on the document containing the current node (not $currentPage), so the trick is to change context with the for-each instruction - something like this:
<!-- Before first template -->
<xsl:key name="subitems-by-id" match="subitem" use="@id" />
<!-- Somewhere after fetching XML doc into $index -->
<xsl:for-each select="$index">
<xsl:apply-templates select="key('subitems-by-id', $id)" />
</xsl:for-each>
<xsl:template match="subitem">
<!-- do stuff -->
</xsl:template>
After a bit more reading on xsl:key and key(), I came across this tidbit that helped explain...
"The xsl:key element alerts the XSLT processor to create an indexed data structure for the key expressions ahead of time. This indexed data structure will be used by the keyfunction."
So even though creating the index takes a bit of time you could easily gain it back when you search for an entry with the key() function. The more nodes you're chewing through the more gain you'll get.
Though I've used keys in the past I learn something new every day!
<!--Fetch the subitem from the index xml. This could also be done using an xpath, but this is faster.-->
<xsl:variable name="tempSubitem">
<xsl:for-each select="$index">
<xsl:copy-of select="key('subitems-by-id', $currentPage/@id)" />
</xsl:for-each>
</xsl:variable>
<!--Convert the subitem into the right format to work with.-->
<xsl:variable name="subitem" select="msxml:node-set($tempSubitem)/*" />
<xsl:variable name="chamber" select="$subitem/ancestor::chamber" />
In the first example I got the Chamber element back, but in the second I don't. Is there a way to get the parent of a node which is fetched with a key?
The reason you can't access the chamber node is because your subitem is nothing but an *evil* clone (you used copy-of to create it) that doesn't belong to the same tree as $currentPage (and thus, the chamber node).
If you do the template thing I did, you'll have access to the entire tree from within that template, and you can create a variable for the chamber node as you did before, e.g.:
<xsl:key name="subitems-by-id" match="subitem" use="@id" />
<!-- Somewhere after fetching XML doc into $index -->
<xsl:for-each select="$index">
<xsl:apply-templates select="key('subitems-by-id', $id)" />
</xsl:for-each>
<xsl:template match="subitem">
<xsl:variable name="chamber" select="ancestor::chamber" />
<!-- This might be quicker if chamber is always 2 steps away: -->
<!-- <xsl:variable name="chamber" select="../.." /> -->
<!-- Do stuff with $chamber -->
</xsl:template>
Thanks for the tip! Only problem is that I want to use the chamber and subitem variable inside multiple templates. In my example I set the variable before any template is called. Can I somehow do your sample without using templates (a bit like my sample)?
xpath slow on large xml
Hello,
I've got a pretty big xml file which is loaded with umbraco.library:GetXmlDocumentByUrl. I use xpath to get an element which is pretty deep in the tree. Here is my xpath:
This seems pretty slow. Is there a way to optimize my code? Can the xpath be faster some other way or can I somehow fragment the xml file more for better performance?
Jeroen
That xpath looks good in that you aren't using '//' or 'descendant-or-self' but are instead specifying exactly how to get to the subitem you want.
It may be that you have a LOT of subitem entries in the XML?
But more likely, the GetXmlDocumentByUrl() is sluggish retrieving a large XML file due to latency on the http request and bandwidth to return the xml itself. If you can, use the optional timeout parameter with GetXmlDocumentByUrl() to reduce the frequency of the request for the xml file itself. And macro caching will also be your friend. You might also want to look at Darren's FeedCache package, which is excellent.
Let us know what you find out.
cheers,
doug.
Hi Doug,
I've been debugging the code and getting the xml with GetXmlDocumentByUrl works fine. The slow part is where I do my xpath. There are a lot of subitem entries indeed. I was hoping that I could improve my xpath somehow. Maybe narrow down in which element I need to search. Thanks for helping.
Jeroen
You can also pass a caching parameter to the GetXmlDocumentByUrl function :-)
Hi Jeroen,
If each step has a lot of items it might help to hand-pick the one you're after (if at all possible) e.g.
- though that's probably not possible for you.
You should try using a key - tricky part is it only works on the document containing the current node (not $currentPage), so the trick is to change context with the for-each instruction - something like this:
<!-- Before first template --> <xsl:key name="subitems-by-id" match="subitem" use="@id" /> <!-- Somewhere after fetching XML doc into $index --> <xsl:for-each select="$index"> <xsl:apply-templates select="key('subitems-by-id', $id)" /> </xsl:for-each> <xsl:template match="subitem"> <!-- do stuff --> </xsl:template>/Chriztian
Hi Chriztian,
I'll try this and see if it works. Is this the best way if I want to fetch only 1 item? On twitter some also said I could use this:
Somehow I think your key technique is better :). I'll let you know when I got it working. Thanks!
Jeroen
Using "//" will definitely be slower. Much slower.
cheers,
doug.
If you're only fetching a single item, the key might not help (curious to know though).
The // thing will likely be the slowest possible solution ;)
/Chriztian
After a bit more reading on xsl:key and key(), I came across this tidbit that helped explain...
"The xsl:key element alerts the XSLT processor to create an indexed data structure for the key expressions ahead of time. This indexed data structure will be used by the keyfunction."
So even though creating the index takes a bit of time you could easily gain it back when you search for an entry with the key() function. The more nodes you're chewing through the more gain you'll get.
Though I've used keys in the past I learn something new every day!
cheers,
doug.
This is how my code is now:
<xsl:key name="subitems-by-id" match="subitem" use="@id" /> <xsl:variable name="tempSubitem"> <xsl:for-each select="$index"> <xsl:copy-of select="key('subitems-by-id', $currentPage/@id)" /> </xsl:for-each> </xsl:variable> <xsl:variable name="subitem" select="msxml:node-set($tempSubitem)/*" />Don't think there is a big performance boost, but I think it's faster :).
Jeroen
Hmm I'm having another problem now:
First I just to got the subItem and than it's parent like this:
Now I've got this:
<!--Fetch the subitem from the index xml. This could also be done using an xpath, but this is faster.--> <xsl:variable name="tempSubitem"> <xsl:for-each select="$index"> <xsl:copy-of select="key('subitems-by-id', $currentPage/@id)" /> </xsl:for-each> </xsl:variable> <!--Convert the subitem into the right format to work with.--> <xsl:variable name="subitem" select="msxml:node-set($tempSubitem)/*" /> <xsl:variable name="chamber" select="$subitem/ancestor::chamber" />In the first example I got the Chamber element back, but in the second I don't. Is there a way to get the parent of a node which is fetched with a key?
Jeroen
Hi Jeroen,
The reason you can't access the chamber node is because your subitem is nothing but an *evil* clone (you used copy-of to create it) that doesn't belong to the same tree as $currentPage (and thus, the chamber node).
If you do the template thing I did, you'll have access to the entire tree from within that template, and you can create a variable for the chamber node as you did before, e.g.:
<xsl:key name="subitems-by-id" match="subitem" use="@id" /> <!-- Somewhere after fetching XML doc into $index --> <xsl:for-each select="$index"> <xsl:apply-templates select="key('subitems-by-id', $id)" /> </xsl:for-each> <xsl:template match="subitem"> <xsl:variable name="chamber" select="ancestor::chamber" /> <!-- This might be quicker if chamber is always 2 steps away: --> <!-- <xsl:variable name="chamber" select="../.." /> --> <!-- Do stuff with $chamber --> </xsl:template>/Chriztian
Hi Chriztian,
Thanks for the tip! Only problem is that I want to use the chamber and subitem variable inside multiple templates. In my example I set the variable before any template is called. Can I somehow do your sample without using templates (a bit like my sample)?
Jeroen
Hi Jeroen,
OK, well - I'd suggest you send the subitem variable along when you're calling other templates, e.g.:
<!-- The subtem template from before --> <xsl:template match="subitem"> <!-- Start your errands here --> <xsl:call-template name="runErrands"> <xsl:with-param name="subitem" select="." /> <xsl:with-param name="chamber" select="../../" /> </xsl:call-template> </xsl:template> <xsl:template name="runErrands"> <xsl:param name="subitem" /> <xsl:param name="chamber" /> <!-- Do stuff --> </xsl:template>You can of course keep calling others, sending the variable/parameters along everytime.
Hope that's useful, then :-)
/Chriztian
Hi Chriztian,
I've applied your sample and it works like a charm :). Thanks for helping!
Jeroen
is working on a reply...