bench.xml 4.72 KB
Newer Older
1
<?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?>
2
<page name="bench">
3
4
5

<title>Benchmarks</title>

6
7
8
<left>
<p>This page briefly presents preliminary benchmarks on the CDuce
prototype.
9
</p>
10
11
12
13
<boxes-toc/>
<p>See also:</p>
<local-links href="index,proto"/>
</left>
14
15

<box title="Methodology" link="meth">
16
<p>
17
18
19
20
The benchmarks were performed on a Linux box, Athlon 750, 128 Mb RAM.
The results are given in seconds. They were measured with
the Unix <tt>time</tt> command. The user time is reported unless
otherwise specified.
21
22
</p>

23
<p>
24
25
These are only <b>preliminary benchmarks</b>, and results
should be taken with much care.
26
27
28
29
30
</p>
</box>

<box title="CDuce vs. XDuce" link="xduce">

31
32
33
34
35
36
37
38
<section title="Addrbook">

<p>
This example was taken from XDuce distribution.
The transformation is simple;
the example is intended to evaluate run-time efficiency.
</p>

39
<p>
40
41
42
<a href="benchmarks/addrbook.cd">CDuce source</a> -
<a href="benchmarks/addrbook.q">XDuce source</a> -
<a href="benchmarks/addrbook.ml">OCaml file</a> (to generate random documents).
43
44
45
46
47
48
49
50
51
52
53
</p>


<table border="1">
 <tr> <td>Addrbook</td> 
      <th>XDuce 0.2.4</th> 
      <th>XDuce 0.2.4 + patopt</th> 
      <th>XDuce 0.4.0 + patopt</th> 
      <th>CDuce</th> </tr>

 <tr align="right"> <th>Type-checking</th> 
54
      <td>0.04</td> <td>0.05</td> <td>0.15</td> <td>0.04</td> </tr>
55
56
57
58
59
 <tr align="right"> <th>Small file (60 Ko)</th>
      <td>1.56</td> <td>0.14</td> <td>9.6</td> <td>0.09</td> </tr>
 <tr align="right"> <th>Medium file (600 Ko)</th>
      <td>260</td> <td>1.16</td> <td>*</td> <td>0.46</td> </tr>
 <tr align ="right"> <th>Large file (6 Mb)</th>
60
      <td>*</td> <td>31.05</td> <td>*</td> <td>6.37</td> </tr>
61
62
63

</table>

64
<p>
65
The * corresponds to checks stopped after ten minutes.
66
67
68
69
70
71
72
73
74
75
76
77
</p>
</section>

<section title="Html2latex">

<p>
This example was taken from XDuce distribution and simplified
(removed support for tables). The transformation is more complex;
the example is intended to evaluate type-checking time.
</p>

<p>
78
79
80
81
<a href="benchmarks/html2latex.cd">CDuce source</a> -
<a href="benchmarks/xhtml-trans.cd">CDuce source (XHTML Transitional DTD)</a> -
<a href="benchmarks/html2latex.q">XDuce source</a> -
<a href="benchmarks/tst_html2latex.xml">Sample input XML file</a>.
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
</p>

<table border="1">
 <tr> <td>Html2latex</td> 
      <th>XDuce 0.2.4</th> 
      <th>XDuce 0.2.4 + patopt</th> 
      <th>XDuce 0.4.0 + patopt</th> 
      <th>CDuce</th> </tr>

 <tr align="right"> <th>Type-checking</th> 
      <td>1.5</td> <td>21.4</td> <td>9.5</td> <td>0.4</td> </tr>
 <tr align="right"> <th>Total</th> 
      <td>2.3</td> <td>26.5</td> <td>10.0</td> <td>0.8</td> </tr>
 </table>

</section>
98

99
100
101
</box>

<box title="CDuce vs. XSLT" link="xslt">
102

103
<p>
104
We also performed benchmarks between CDuce and a fast XSLT processor (the
105
<tt>xsltproc</tt> program from <a href="http://xmlsoft.org/XSLT/">
106
Gnome libxslt library</a>).
107
</p>
108
109

<p>
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
The example is intended to evaluate run-time efficiency. Even though
the transformation is quite complex, type-checking time for CDuce 
is negligible. A few remarks:
</p>
<ul>
<li>type-based optimization
brings almost no gain (only tag checks can be avoided);</li>
<li>the CDuce program must validate its input document (the XSLT
stylesheet can operate on any document);</li>
<li>the XSLT processor is written in C; the CDuce interpreter
is written in Objective Caml.</li>
</ul>

<p>
Taking these remarks into account, the results below are more than
encouraging.
</p>

<p>
129
130
131
132
<a href="benchmarks/split.cd">CDuce source</a> -
<a href="benchmarks/split.xslt1.xml">XSLT source</a> -
<a href="benchmarks/split.xslt2.xml">XSLT source (another version)</a> -
<a href="benchmarks/split.ml">OCaml file</a> (to generate random documents).
133
134
135
136
</p>


<table border="1">
137
 <tr> <td>split</td> 
138
139
140
141
142
      <th>XSLT (1)</th> 
      <th>XSLT (2)</th> 
      <th>CDuce</th> </tr>

 <tr align="right"> <th>5 Mb</th> 
143
      <td>8.90</td> <td>8.05</td> <td>5.30</td> </tr>
144
 <tr align="right"> <th>27 Mb</th>
145
      <td>47.10</td> <td>68.40</td> <td>33.85</td> </tr>
146
147
</table>

148
149
150
151
152
153
154
155
156
<p>
The table above displays CPU user time (parsing + transformation) .
The table below, instead, displays total wall-clock time
(including system time). The results indicate that CDuce
behave much better when the XML document
does'nt fit in memory (and swapping occurs).
</p>

<table border="1">
157
 <tr> <td>split</td> 
158
159
160
161
162
163
164
165
166
      <th>XSLT (1)</th> 
      <th>XSLT (2)</th> 
      <th>CDuce</th> </tr>

 <tr align="right"> <th>5 Mb</th> 
      <td>9.05</td> <td>8.26</td> <td>5.40</td> </tr>
 <tr align="right"> <th>27 Mb</th>
      <td>320.80</td> <td>324.50</td> <td>83.65</td> </tr>
</table>
167

168
169
</box>

170
<box title="CDuce vs. XQuery" link="xquery">
171
172
173
174

<p>
<b>A benchmark test with the <a href="http://www.w3.org/TR/xmlquery-use-cases"> XQuery Use Cases</a> is in preparation</b>
</p>
175
</box>
176
</page>