diff --git a/AGENTS.md b/AGENTS.md index f546913..5825a69 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,6 +13,8 @@ generated parser, on netstandard2.0): keep it so, and keep containers and other |---|---| | `src/SharpEquations/` | The library: `Public/` (the public API), `Internal/` (one folder per format), `Resources/` (XSLT stylesheets) | | `src/SharpEquations.RegexPolyfill/` | Build time only: the `[GeneratedRegex]` source generator of the netstandard2.0 build | +| `src/SharpEquations.XsltGenerator/` | Build time only: the source generator that compiles the XSLT stylesheets of `Resources/` to C# | +| `benchmarks/SharpEquations.Benchmarks/` | BenchmarkDotNet benchmarks of reading, writing and converting the corpora | | `tests/SharpEquations.Tests/` | TUnit tests, in folders by subject (PublicApi, MathML, Omml, Mtef, UnicodeMath, LaTeX, Property, Regression) | | `tests/data/` | Test data and its provenance (`tests/data/README.md`) | @@ -58,8 +60,14 @@ init-only record: `Display`, `Strict`, `PreserveFormatting`, `LaTeXFlavor` stand UnicodeMath writers. Content MathML of the source (`apply`, `ci`, `cn`..., outside the annotations) is first rewritten as presentation MathML by `ctop.xsl` (David Carlisle's, see `NOTICE`), keeping the presentation MathML around it. - `Internal/Omml/`: OMML ↔ MathML with the XSLT 1.0 stylesheets of `Resources/` (from the TEI Stylesheets, see - `NOTICE`), run with `XslCompiledTransform`. OMML is written as Word writes it (plain letters with `m:sty`/`m:scr`, + `NOTICE`). OMML is written as Word writes it (plain letters with `m:sty`/`m:scr`, underbars as `m:bar`), and an upright word of OMML (`max`) is read as one identifier. +- `Internal/Xslt/`: the run time of the stylesheets, which `SharpEquations.XsltGenerator` compiles to C# at build time + (no code generated at run time, so the library runs under Native AOT, `IsAotCompatible` on net10.0): the XPath data + model over LINQ to XML (`XPathFunctions`), the result tree (`XsltOutput`), and the partial classes the generator + implements (`Stylesheets.cs`; the generated code goes to `obj/`, and is declared there so that Parlot, which compiles + the project without the other generators, sees it). `Xslt/CompiledStylesheetTests` compares them with + `XslCompiledTransform` on the corpora. - `Internal/Mtef/`: MTEF of Equation Editor 3.0 and MathType, read only; a C# port of the jure/mathtype gem, converted with `mtef2mml.xsl`. It reads the "Equation Native" stream of the OLE object; opening the OLE object is the caller's. - `Internal/UnicodeMath/`: a port of UnicodeMathML. Its PEG grammar is `UnicodeMathGrammar.parlot.cs`, compiled at diff --git a/BuildVersion.props b/BuildVersion.props index c05892f..c9ed6c6 100644 --- a/BuildVersion.props +++ b/BuildVersion.props @@ -2,7 +2,7 @@ - 1.0.7 + 1.1.0 diff --git a/SharpEquations.slnx b/SharpEquations.slnx index 7205bd2..e10d9a6 100644 --- a/SharpEquations.slnx +++ b/SharpEquations.slnx @@ -11,6 +11,7 @@ + diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 9cfba58..0dab68c 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -78,6 +78,32 @@ The XSLT stylesheets recurse over the tree, and a stack overflow ends the proces caught, so `EquationStylesheets` rejects input nested more than 512 levels deep (`MaxDepth`). The MTEF reader stops at 256 nested record lists for the same reason. +## The stylesheets compiled to C# (`src/SharpEquations.XsltGenerator/`, `Internal/Xslt/`) + +The three XSLT 1.0 stylesheets of `Resources/` (`omml2mml.xsl`, `mml2omml.xsl`, `mtef2mml.xsl`) are compiled to C# at +build time by `SharpEquations.XsltGenerator`, a source generator that reads them as additional files. Nothing is +generated at run time, unlike `XslCompiledTransform`, which emits IL: so the library runs under Native AOT, and the +stylesheets are not embedded in it. + +- Each stylesheet becomes a class named after its file (`Omml2mmlStylesheet`), declared partial in + `Internal/Xslt/Stylesheets.cs`, with a method for each template, a lazy property for each global variable and, for + each mode, the dispatch of `xsl:apply-templates`: by the name of the element first, then the rules in the order of + XSLT (highest priority, then the last in the stylesheet), then the built-in templates. +- Each XPath expression becomes a C# expression over `XPathFunctions` (the data model of XPath over LINQ to XML, its + functions and conversions). Its type is known when it can be (a string, a number, a boolean, a node-set, a result + tree fragment), and is `object` otherwise (a parameter). +- `XsltOutput` builds the result as LINQ to XML, as `XslCompiledTransform` gave it when its output was written as text + and parsed again: text written with `disable-output-escaping` is markup, with the text next to it (`mtef2mml.xsl` + writes character references in pieces), and empty text leaves `` rather than ``. +- Strings are counted, cut and translated by code points, as XPath defines them (`XslCompiledTransform` counted UTF-16 + units). +- What the stylesheets do not use (keys, `xsl:sort`, `xsl:number`, imports, extension functions, the `following` and + `preceding` axes) is an error of the build (XSLT001). + +`Xslt/CompiledStylesheetTests` checks that the compiled stylesheets give what `XslCompiledTransform` gives for the +corpora: the OMML of Word documents and of the UnicodeMathML examples, their MathML, and the MTEF of the test data. It +leaves out the inputs with characters outside the basic plane, where the two count differently. + ## The normalized MathML (`Internal/MathML/MathMLParser.cs`) `Normalize` is what makes equality and round trips possible. The same mathematics read from different formats, or @@ -146,15 +172,15 @@ an `annotation-xml` is left alone, and a `semantics` whose first child is Conten ### OMML (`Internal/Omml/`) OMML ↔ MathML uses the XSLT 1.0 stylesheets of the TEI Consortium (`Resources/omml2mml.xsl`, `Resources/mml2omml.xsl`, -BSD-2-Clause), the descendants of the ones Microsoft ships with Office. `EquationStylesheets` compiles them once -with `XslCompiledTransform`. C# code before and after each stylesheet fixes what the stylesheets get wrong or what -Word writes differently. +BSD-2-Clause), the descendants of the ones Microsoft ships with Office, compiled to C# at build time (see "The +stylesheets compiled to C#" below). C# code before and after each stylesheet fixes what the stylesheets get wrong or +what Word writes differently. **Reading (`OmmlParser`):** - `m:oMathPara` is a display equation, `m:oMath` an inline one. - `ST_OnOff` values are normalized, because the stylesheet only knows `on`/`off`. -- Characters outside the basic plane go through the stylesheet as private use characters, because it splits text into - UTF-16 units. +- Characters outside the basic plane go through the stylesheet as private use characters, because + `XslCompiledTransform`, which ran it before it was compiled to C#, split text into UTF-16 units. - Lines aligned with `m:aln` become a table marked as aligned lines. - Numbers split across runs are joined. - Word's equation numbers (`#(1)`) become `mlabeledtr`. diff --git a/src/SharpEquations.XsltGenerator/AnalyzerReleases.Shipped.md b/src/SharpEquations.XsltGenerator/AnalyzerReleases.Shipped.md new file mode 100644 index 0000000..f50bb1f --- /dev/null +++ b/src/SharpEquations.XsltGenerator/AnalyzerReleases.Shipped.md @@ -0,0 +1,2 @@ +; Shipped analyzer releases +; https://github.com/dotnet/roslyn-analyzers/blob/main/src/Microsoft.CodeAnalysis.Analyzers/ReleaseTrackingAnalyzers.Help.md diff --git a/src/SharpEquations.XsltGenerator/AnalyzerReleases.Unshipped.md b/src/SharpEquations.XsltGenerator/AnalyzerReleases.Unshipped.md new file mode 100644 index 0000000..0ff5f27 --- /dev/null +++ b/src/SharpEquations.XsltGenerator/AnalyzerReleases.Unshipped.md @@ -0,0 +1,8 @@ +; Unshipped analyzer release +; https://github.com/dotnet/roslyn-analyzers/blob/main/src/Microsoft.CodeAnalysis.Analyzers/ReleaseTrackingAnalyzers.Help.md + +### New Rules + +Rule ID | Category | Severity | Notes +--------|----------|----------|------- +XSLT001 | SharpEquations.Xslt | Error | XsltGenerator, a stylesheet that cannot be compiled diff --git a/src/SharpEquations.XsltGenerator/CodeWriter.cs b/src/SharpEquations.XsltGenerator/CodeWriter.cs new file mode 100644 index 0000000..537b48d --- /dev/null +++ b/src/SharpEquations.XsltGenerator/CodeWriter.cs @@ -0,0 +1,43 @@ +namespace SharpEquations.XsltGenerator +{ + using System.Text; + + internal sealed class CodeWriter + { + private readonly StringBuilder text = new StringBuilder(); + private int indent; + + public CodeWriter(int indent = 0) => this.indent = indent; + + public CodeWriter Line(string line = "") + { + if (line.Length > 0) + text.Append(' ', indent * 4).Append(line); + text.Append('\n'); + return this; + } + + public CodeWriter Open(string? line = null) + { + if (line != null) + Line(line); + Line("{"); + indent++; + return this; + } + + public CodeWriter Close(string suffix = "") + { + indent--; + return Line("}" + suffix); + } + + public CodeWriter Append(CodeWriter other) + { + text.Append(other.text); + return this; + } + + public override string ToString() => text.ToString(); + } +} diff --git a/src/SharpEquations.XsltGenerator/SharpEquations.XsltGenerator.csproj b/src/SharpEquations.XsltGenerator/SharpEquations.XsltGenerator.csproj new file mode 100644 index 0000000..97e4b3c --- /dev/null +++ b/src/SharpEquations.XsltGenerator/SharpEquations.XsltGenerator.csproj @@ -0,0 +1,20 @@ + + + + + netstandard2.0 + enable + true + true + false + + + + + + + + + diff --git a/src/SharpEquations.XsltGenerator/StylesheetCompiler.cs b/src/SharpEquations.XsltGenerator/StylesheetCompiler.cs new file mode 100644 index 0000000..605164e --- /dev/null +++ b/src/SharpEquations.XsltGenerator/StylesheetCompiler.cs @@ -0,0 +1,1011 @@ +namespace SharpEquations.XsltGenerator +{ + using System.Collections.Generic; + using System.Linq; + using System.Text; + using System.Xml; + using System.Xml.Linq; + + /// + /// An XSLT 1.0 stylesheet compiled to a C# class: a method for each template, the dispatch of + /// xsl:apply-templates for each mode (by the name of the element first), a lazy property for each + /// global variable. What the stylesheets of SharpEquations do not use (keys, sorting, numbering, imports, + /// extension functions) is reported as an error. + /// + internal sealed class StylesheetCompiler + { + private static readonly XNamespace Xsl = "http://www.w3.org/1999/XSL/Transform"; + + private readonly XElement stylesheet; + private readonly string className; + private readonly string fileName; + private readonly XPathCompiler xpath; + + private readonly CodeWriter nameFields = new CodeWriter(2); + private readonly CodeWriter testFields = new CodeWriter(2); + private readonly CodeWriter staticFields = new CodeWriter(2); + private readonly CodeWriter instanceFields = new CodeWriter(2); + private readonly CodeWriter members = new CodeWriter(2); + + private readonly Dictionary names = new Dictionary(); + private readonly Dictionary tests = new Dictionary(); + private readonly Dictionary namedTemplates = new Dictionary(); + private readonly Dictionary globals = new Dictionary(); + private readonly Dictionary modes = new Dictionary { [""] = 0 }; + private readonly List rules = new List(); + private int counter; + + private sealed class NamedTemplate + { + public NamedTemplate(string method, List parameters) + { + Method = method; + Parameters = parameters; + } + + public string Method { get; } + + public List Parameters { get; } + } + + private sealed class GlobalVariable + { + public GlobalVariable(XElement element) => Element = element; + + public XElement Element { get; } + + public Variable? Variable { get; set; } + + public bool Compiling { get; set; } + } + + private enum Bucket + { + Root, + Element, + NamedElement, + Attribute, + Text, + Comment, + ProcessingInstruction, + } + + private sealed class PatternStep + { + public PatternStep(Step step, bool afterDescendant) + { + Step = step; + AfterDescendant = afterDescendant; + } + + public Step Step { get; } + + /// Whether the step follows // rather than /. + public bool AfterDescendant { get; } + } + + private sealed class Rule + { + public Rule(int index, XElement template, string method, int mode, double priority, bool isRoot, bool absolute, List steps) + { + Index = index; + Template = template; + Method = method; + Mode = mode; + Priority = priority; + IsRoot = isRoot; + Absolute = absolute; + Steps = steps; + } + + public int Index { get; } + + public XElement Template { get; } + + public string Method { get; } + + public int Mode { get; } + + public double Priority { get; } + + public bool IsRoot { get; } + + public bool Absolute { get; } + + public List Steps { get; } + + public Step? Last => Steps.Count == 0 ? null : Steps[Steps.Count - 1].Step; + } + + public StylesheetCompiler(XElement stylesheet, string className, string fileName) + { + this.stylesheet = stylesheet; + this.className = className; + this.fileName = fileName; + xpath = new XPathCompiler(this); + } + + public string NewName(string prefix) => prefix + (++counter); + + public static XsltException Error(XObject node, string message) + { + var info = (IXmlLineInfo)node; + return new XsltException(message) { Line = info.HasLineInfo() ? info.LineNumber : 0, Column = info.HasLineInfo() ? info.LinePosition : 0 }; + } + + // static fields: names, node tests, tables + + public string Name(XName name) + { + var key = name.ToString(); + if (!names.TryGetValue(key, out var field)) + { + field = "Name" + names.Count; + names.Add(key, field); + nameFields.Line("private static readonly XName " + field + " = XName.Get(" + XPathCompiler.Literal(name.LocalName) + ", " + XPathCompiler.Literal(name.NamespaceName) + ");"); + } + return field; + } + + public string NodeTest(NodeTestSyntax test, XElement scope) + { + string key; + string init; + switch (test.Kind) + { + case TestKind.Name: + { + var name = XName.Get(test.Local!, Namespace(test.Prefix, scope)); + key = "name " + name; + init = "NodeTest.Named(" + Name(name) + ")"; + break; + } + case TestKind.Namespace: + { + var ns = Namespace(test.Prefix, scope); + key = "ns " + ns; + init = "NodeTest.InNamespace(" + XPathCompiler.Literal(ns) + ")"; + break; + } + case TestKind.AnyName: + return "NodeTest.AnyName"; + case TestKind.Node: + return "NodeTest.Node"; + case TestKind.Text: + return "NodeTest.Text"; + case TestKind.Comment: + return "NodeTest.Comment"; + default: + key = "pi " + test.Local; + init = "NodeTest.ProcessingInstruction(" + (test.Local == null ? "null" : XPathCompiler.Literal(test.Local)) + ")"; + break; + } + if (!tests.TryGetValue(key, out var field)) + { + field = "Test" + tests.Count; + tests.Add(key, field); + testFields.Line("private static readonly NodeTest " + field + " = " + init + ";"); + } + return field; + } + + private static string Namespace(string? prefix, XElement scope) + { + if (prefix == null) + return ""; + var ns = scope.GetNamespaceOfPrefix(prefix); + if (ns == null) + throw Error(scope, "The prefix '" + prefix + "' is not declared."); + return ns.NamespaceName; + } + + // the stylesheet + + public string Compile() + { + if (stylesheet.Name != Xsl + "stylesheet" && stylesheet.Name != Xsl + "transform") + throw Error(stylesheet, "Not an XSLT stylesheet."); + + var templates = new List(); + foreach (var element in stylesheet.Elements()) + { + if (element.Name.Namespace != Xsl) + continue; + switch (element.Name.LocalName) + { + case "template": + templates.Add(element); + break; + case "variable": + case "param": + globals[Attr(element, "name", true)!] = new GlobalVariable(element); + break; + case "output": + break; + default: + throw Error(element, "xsl:" + element.Name.LocalName + " is not supported."); + } + } + + foreach (var mode in stylesheet.Descendants().Where(e => e.Name == Xsl + "apply-templates" || e.Name == Xsl + "template")) + ModeIndex((string?)mode.Attribute("mode")); + + for (var i = 0; i < templates.Count; i++) + { + var name = Attr(templates[i], "name", false); + if (name != null) + namedTemplates[name] = new NamedTemplate("Named" + i, TemplateParameters(templates[i]).Select(p => Attr(p, "name", true)!).ToList()); + } + + for (var i = 0; i < templates.Count; i++) + CompileTemplate(templates[i], i); + + for (var mode = 0; mode < modes.Count; mode++) + CompileDispatch(mode); + + return Assemble(); + } + + private int ModeIndex(string? mode) + { + var key = mode ?? ""; + if (!modes.TryGetValue(key, out var index)) + { + index = modes.Count; + modes.Add(key, index); + } + return index; + } + + private static string? Attr(XElement element, string name, bool required) + { + var value = (string?)element.Attribute(name); + if (value == null && required) + throw Error(element, "xsl:" + element.Name.LocalName + " needs the attribute " + name + "."); + return value; + } + + private static IEnumerable TemplateParameters(XElement template) => + template.Elements().TakeWhile(e => e.Name == Xsl + "param"); + + private string Assemble() + { + var file = new CodeWriter(); + file.Line("// ") + .Line("// " + fileName + ", compiled by SharpEquations.XsltGenerator") + .Line("#nullable enable") + .Line("#pragma warning disable") + .Line() + .Open("namespace SharpEquations.Internal.Xslt") + .Line("using System.Collections.Generic;") + .Line("using System.Xml.Linq;") + .Line("using static SharpEquations.Internal.Xslt.XPathFunctions;") + .Line() + .Line("// the stylesheet " + fileName + ", compiled to C#") + .Open("internal sealed partial class " + className); + file.Append(nameFields).Append(testFields).Append(staticFields); + file.Line() + .Line("private readonly XsltOutput output = new XsltOutput();") + .Line("private readonly XObject root;") + .Append(instanceFields) + .Line() + .Line("private " + className + "(XObject root) => this.root = root;") + .Line() + .Line("/// Runs the stylesheet on the document of the element (the element itself, unless it has a parent).") + .Open("public static partial XElement? Transform(XElement input)") + .Line("var document = new XDocument(input);") + .Line("var stylesheet = new " + className + "(document);") + .Line("stylesheet.Apply0(Single(document), null);") + .Line("return stylesheet.output.Result();") + .Close(); + file.Append(members); + file.Close().Close(); + return file.ToString(); + } + + // templates + + private void CompileTemplate(XElement template, int index) + { + var name = Attr(template, "name", false); + var match = Attr(template, "match", false); + if (name == null && match == null) + throw Error(template, "A template needs a name or a match pattern."); + + var parameters = TemplateParameters(template).ToList(); + if (name != null) + { + var named = namedTemplates[name]; + var signature = string.Concat(named.Parameters.Select((p, i) => ", object? a" + i)); + var w = new CodeWriter(2); + w.Line().Open("private void " + named.Method + "(XObject n0, int p0, int s0" + signature + ")"); + CompileBody(template, parameters, i => "a" + i, w); + w.Close(); + members.Append(w); + } + + if (match == null) + return; + + var method = "Template" + index; + var body = new CodeWriter(2); + body.Line().Open("private void " + method + "(XObject n0, int p0, int s0, TemplateArguments? args)"); + if (name != null) + { + var named = namedTemplates[name]; + body.Line(named.Method + "(n0, p0, s0" + string.Concat(named.Parameters.Select(p => ", args?.Get(" + XPathCompiler.Literal(p) + ")")) + ");"); + } + else + { + var names = parameters.Select(p => Attr(p, "name", true)!).ToList(); + CompileBody(template, parameters, i => "args?.Get(" + XPathCompiler.Literal(names[i]) + ")", body); + } + body.Close(); + members.Append(body); + + var mode = ModeIndex((string?)template.Attribute("mode")); + var priority = (string?)template.Attribute("priority"); + foreach (var alternative in Alternatives(XPathParser.Parse(match), template)) + { + var steps = new List(); + var afterDescendant = false; + foreach (var step in alternative.Steps) + { + if (step.IsDescendantOrSelfNode) + { + afterDescendant = true; + continue; + } + if (step.Axis != "child" && step.Axis != "attribute") + throw Error(template, "The axis " + step.Axis + " cannot be used in a pattern."); + steps.Add(new PatternStep(step, afterDescendant)); + afterDescendant = false; + } + var isRoot = alternative.Absolute && steps.Count == 0; + // a leading // anchors nothing: the pattern matches at any depth + var absolute = alternative.Absolute && steps.Count > 0 && !steps[0].AfterDescendant; + var rule = new Rule(rules.Count, template, method, mode, + priority != null ? double.Parse(priority, System.Globalization.CultureInfo.InvariantCulture) : DefaultPriority(alternative, steps), + isRoot, absolute, steps); + rules.Add(rule); + CompileMatch(rule); + } + } + + private static IEnumerable Alternatives(Expr pattern, XElement template) + { + switch (pattern) + { + case BinaryExpr union when union.Op == "|": + return Alternatives(union.Left, template).Concat(Alternatives(union.Right, template)); + case PathExpr path when path.Start == null: + return new[] { path }; + default: + throw Error(template, "Not a pattern: " + (string?)template.Attribute("match")); + } + } + + private static double DefaultPriority(PathExpr alternative, List steps) + { + if (alternative.Absolute || steps.Count != 1 || steps[0].Step.Predicates.Count > 0) + return 0.5; + var test = steps[0].Step.Test; + switch (test.Kind) + { + case TestKind.Name: + return 0; + case TestKind.Namespace: + return -0.25; + case TestKind.ProcessingInstruction when test.Local != null: + return 0; + default: + return -0.5; + } + } + + private void CompileBody(XElement template, List parameters, System.Func argument, CodeWriter w) + { + var scope = new Scope(null); + var context = new Context("n0", "p0", "s0", "n0", scope, template); + for (var i = 0; i < parameters.Count; i++) + { + var parameter = parameters[i]; + var id = NewName("v"); + var paramContext = context.WithElement(parameter); + var select = Attr(parameter, "select", false); + if (select != null) + { + w.Line("var " + id + " = " + argument(i) + " ?? " + XPathCompiler.Box(xpath.Compile(select, paramContext)) + ";"); + } + else if (HasContent(parameter)) + { + w.Line("var " + id + " = " + argument(i) + ";"); + w.Open("if (" + id + " == null)"); + w.Line("output.BeginCapture();"); + CompileSequence(parameter.Nodes(), paramContext, w); + w.Line(id + " = output.EndFragment();"); + w.Close(); + } + else + { + w.Line("var " + id + " = " + argument(i) + " ?? \"\";"); + } + scope.Add(Attr(parameter, "name", true)!, new Variable(id, XType.Any)); + } + CompileSequence(template.Nodes().Where(n => !(n is XElement e && parameters.Contains(e))), context, w); + } + + private static bool HasContent(XElement element) => + element.Nodes().Any(n => n is XElement || (n is XText t && t.Value.Trim().Length > 0)); + + // patterns + + private void CompileMatch(Rule rule) + { + var w = new CodeWriter(2); + var context = new Context("node", "1", "1", "node", new Scope(null), rule.Template); + if (rule.IsRoot) + { + w.Line().Line("private static bool Match" + rule.Index + "(XObject node) => node is XDocument;"); + members.Append(w); + return; + } + + for (var i = 0; i < rule.Steps.Count; i++) + { + var step = rule.Steps[i].Step; + var attributeAxis = step.Axis == "attribute"; + var test = NodeTest(step.Test, rule.Template) + ".Matches(node, " + (attributeAxis ? "true" : "false") + ")"; + if (step.Predicates.Count > 0) + { + if (step.Predicates.Any(p => xpath.IsPositional(p, context))) + test = "MatchesStep(node, Axis." + (attributeAxis ? "Attribute" : "Child") + ", " + NodeTest(step.Test, rule.Template) + ", " + + string.Join(", ", step.Predicates.Select(p => xpath.Predicate(p, context))) + ")"; + else + test += string.Concat(step.Predicates.Select(p => " && " + XPathCompiler.Bool(xpath.Compile(p, context)))); + } + + w.Line().Open("private bool Match" + rule.Index + "_" + i + "(XObject node)"); + w.Line("if (!(" + test + "))").Line(" return false;"); + if (i == 0) + { + w.Line(rule.Absolute ? "return Parent(node) is XDocument;" : "return true;"); + } + else if (rule.Steps[i].AfterDescendant) + { + w.Line("for (var ancestor = Parent(node); ancestor != null; ancestor = Parent(ancestor))"); + w.Open().Line("if (Match" + rule.Index + "_" + (i - 1) + "(ancestor))").Line(" return true;").Close(); + w.Line("return false;"); + } + else + { + w.Line("var parent = Parent(node);"); + w.Line("return parent != null && Match" + rule.Index + "_" + (i - 1) + "(parent);"); + } + w.Close(); + } + w.Line().Line("private bool Match" + rule.Index + "(XObject node) => Match" + rule.Index + "_" + (rule.Steps.Count - 1) + "(node);"); + members.Append(w); + } + + private static Bucket[] Buckets(Rule rule) + { + if (rule.IsRoot) + return new[] { Bucket.Root }; + var last = rule.Last!; + if (last.Axis == "attribute") + return new[] { Bucket.Attribute }; + switch (last.Test.Kind) + { + case TestKind.Name: + return new[] { Bucket.NamedElement }; + case TestKind.Namespace: + case TestKind.AnyName: + return new[] { Bucket.Element }; + case TestKind.Node: + return new[] { Bucket.Element, Bucket.Text, Bucket.Comment, Bucket.ProcessingInstruction }; + case TestKind.Text: + return new[] { Bucket.Text }; + case TestKind.Comment: + return new[] { Bucket.Comment }; + default: + return new[] { Bucket.ProcessingInstruction }; + } + } + + private void CompileDispatch(int mode) + { + // the rule that wins is the one of highest priority and, among those, the last in the stylesheet + var ordered = rules.Where(r => r.Mode == mode).OrderByDescending(r => r.Priority).ThenByDescending(r => r.Index).ToList(); + List In(Bucket bucket) => ordered.Where(r => Buckets(r).Contains(bucket)).ToList(); + + var w = new CodeWriter(2); + w.Line().Open("private void Apply" + mode + "(List nodes, TemplateArguments? args)"); + w.Line("for (var i = 0; i < nodes.Count; i++)").Line(" Dispatch" + mode + "(nodes[i], i + 1, nodes.Count, args);"); + w.Close(); + + w.Line().Open("private void Dispatch" + mode + "(XObject node, int position, int size, TemplateArguments? args)"); + w.Open("switch (node)"); + + w.Line("case XElement element:").Open(); + var named = ordered.Where(r => Buckets(r).Contains(Bucket.NamedElement)).ToList(); + var elementNames = named.Select(r => XName.Get(r.Last!.Test.Local!, Namespace(r.Last.Test.Prefix, r.Template))).Distinct().ToList(); + var generic = In(Bucket.Element); + if (elementNames.Count > 0) + { + var table = "Elements" + mode; + staticFields.Line("private static readonly Dictionary " + table + " = new Dictionary") + .Line("{"); + for (var i = 0; i < elementNames.Count; i++) + staticFields.Line(" [" + Name(elementNames[i]) + "] = " + i + ","); + staticFields.Line("};"); + w.Open("switch (" + table + ".TryGetValue(element.Name, out var index) ? index : -1)"); + for (var i = 0; i < elementNames.Count; i++) + { + var name = elementNames[i]; + w.Line("case " + i + ":").Open(); + var candidates = ordered.Where(r => generic.Contains(r) + || (named.Contains(r) && XName.Get(r.Last!.Test.Local!, Namespace(r.Last.Test.Prefix, r.Template)) == name)); + EmitCandidates(candidates, w); + w.Line("break;").Close(); + } + w.Line("default:").Open(); + EmitCandidates(generic, w); + w.Line("break;").Close(); + w.Close(); + } + else + { + EmitCandidates(generic, w); + } + w.Line("Apply" + mode + "(ChildNodes(node), null);").Line("return;").Close(); + + w.Line("case XDocument _:").Open(); + EmitCandidates(In(Bucket.Root), w); + w.Line("Apply" + mode + "(ChildNodes(node), null);").Line("return;").Close(); + + w.Line("case XAttribute _:").Open(); + EmitCandidates(In(Bucket.Attribute), w); + w.Line("output.Text(StringValue(node));").Line("return;").Close(); + + w.Line("case XText _:").Open(); + EmitCandidates(In(Bucket.Text), w); + w.Line("output.Text(StringValue(node));").Line("return;").Close(); + + w.Line("case XComment _:").Open(); + EmitCandidates(In(Bucket.Comment), w); + w.Line("return;").Close(); + + w.Line("case XProcessingInstruction _:").Open(); + EmitCandidates(In(Bucket.ProcessingInstruction), w); + w.Line("return;").Close(); + + w.Close().Close(); + members.Append(w); + } + + private static void EmitCandidates(IEnumerable candidates, CodeWriter w) + { + foreach (var rule in candidates) + { + w.Open("if (Match" + rule.Index + "(node))"); + w.Line(rule.Method + "(node, position, size, args);").Line("return;"); + w.Close(); + } + } + + // global variables and parameters + + public Variable? Global(string name) + { + if (!globals.TryGetValue(name, out var global)) + return null; + if (global.Variable != null) + return global.Variable; + if (global.Compiling) + throw Error(global.Element, "The global variable $" + name + " depends on itself."); + global.Compiling = true; + + var id = "G" + NewName("_"); + var element = global.Element; + var context = new Context("n0", "p0", "s0", "n0", new Scope(null), element); + var w = new CodeWriter(2); + var select = Attr(element, "select", false); + XType type; + w.Line(); + if (select != null) + { + var value = xpath.Compile(select, context); + type = value.Type; + w.Open("private " + XPathCompiler.TypeName(type) + " Compute" + id + "()"); + w.Line("XObject n0 = root;").Line("const int p0 = 1, s0 = 1;"); + w.Line("return " + value.Code + ";"); + } + else if (HasContent(element)) + { + type = XType.Fragment; + w.Open("private ResultFragment Compute" + id + "()"); + w.Line("XObject n0 = root;").Line("const int p0 = 1, s0 = 1;"); + w.Line("output.BeginCapture();"); + CompileSequence(element.Nodes(), context, w); + w.Line("return output.EndFragment();"); + } + else + { + type = XType.String; + w.Open("private string Compute" + id + "()"); + w.Line("return \"\";"); + } + w.Close(); + + var typeName = XPathCompiler.TypeName(type); + instanceFields.Line("private bool " + id + "Done;").Line("private " + typeName + " " + id + "Value = default!;"); + w.Line().Open("private " + typeName + " " + id) + .Open("get") + .Open("if (!" + id + "Done)") + .Line(id + "Value = Compute" + id + "();") + .Line(id + "Done = true;") + .Close() + .Line("return " + id + "Value;") + .Close() + .Close(); + members.Append(w); + global.Variable = new Variable(id, type); + return global.Variable; + } + + // instructions + + private void CompileSequence(IEnumerable nodes, Context context, CodeWriter w) + { + context = context.WithScope(new Scope(context.Scope)); + foreach (var node in nodes) + { + switch (node) + { + case XText text when text.Value.Trim().Length == 0: + break; + case XText text: + w.Line("output.Text(" + XPathCompiler.Literal(text.Value) + ");"); + break; + case XElement element: + try + { + if (element.Name.Namespace == Xsl) + CompileInstruction(element, context.WithElement(element), w); + else + CompileLiteralElement(element, context.WithElement(element), w); + } + catch (XsltException e) when (e.Line == 0) + { + var info = (IXmlLineInfo)element; + e.Line = info.LineNumber; + e.Column = info.LinePosition; + throw; + } + break; + } + } + } + + private void CompileLiteralElement(XElement element, Context context, CodeWriter w) + { + w.Line("output.StartElement(" + Name(element.Name) + ");"); + foreach (var attribute in element.Attributes()) + { + if (attribute.IsNamespaceDeclaration) + continue; + if (attribute.Name.Namespace == Xsl) + { + if (attribute.Name.LocalName == "use-attribute-sets") + throw Error(attribute, "xsl:use-attribute-sets is not supported."); + continue; + } + w.Line("output.Attribute(" + Name(attribute.Name) + ", " + XPathCompiler.Str(AttributeValueTemplate(attribute.Value, context)) + ");"); + } + CompileSequence(element.Nodes(), context, w); + w.Line("output.EndElement();"); + } + + private void CompileInstruction(XElement element, Context context, CodeWriter w) + { + switch (element.Name.LocalName) + { + case "apply-templates": + { + var select = Attr(element, "select", false); + var nodes = select == null ? "ChildNodes(" + context.Node + ")" : XPathCompiler.Nodes(xpath.Compile(select, context)); + var args = "null"; + var parameters = WithParameters(element, context, w); + if (parameters.Count > 0) + { + args = NewName("args"); + w.Line("var " + args + " = new TemplateArguments(new[] { " + string.Join(", ", parameters.Keys.Select(XPathCompiler.Literal)) + + " }, new object[] { " + string.Join(", ", parameters.Values) + " });"); + } + w.Line("Apply" + ModeIndex((string?)element.Attribute("mode")) + "(" + nodes + ", " + args + ");"); + break; + } + case "call-template": + { + var name = Attr(element, "name", true)!; + if (!namedTemplates.TryGetValue(name, out var template)) + throw Error(element, "There is no template named " + name + "."); + var parameters = WithParameters(element, context, w); + var args = string.Concat(template.Parameters.Select(p => ", " + (parameters.TryGetValue(p, out var value) ? value : "null"))); + w.Line(template.Method + "(" + context.Node + ", " + context.Position + ", " + context.Size + args + ");"); + break; + } + case "value-of": + { + var value = XPathCompiler.Str(xpath.Compile(Attr(element, "select", true)!, context)); + w.Line((DisablesEscaping(element) ? "output.RawText(" : "output.Text(") + value + ");"); + break; + } + case "text": + if (element.Value.Length > 0) + w.Line((DisablesEscaping(element) ? "output.RawText(" : "output.Text(") + XPathCompiler.Literal(element.Value) + ");"); + break; + case "if": + w.Open("if (" + XPathCompiler.Bool(xpath.Compile(Attr(element, "test", true)!, context)) + ")"); + CompileSequence(element.Nodes(), context, w); + w.Close(); + break; + case "choose": + { + var first = true; + foreach (var branch in element.Elements()) + { + switch (branch.Name.LocalName) + { + case "when" when branch.Name.Namespace == Xsl: + var test = XPathCompiler.Bool(xpath.Compile(Attr(branch, "test", true)!, context.WithElement(branch))); + w.Open((first ? "if (" : "else if (") + test + ")"); + break; + case "otherwise" when branch.Name.Namespace == Xsl && !first: + w.Open("else"); + break; + case "otherwise" when branch.Name.Namespace == Xsl: + w.Open(); + break; + default: + throw Error(branch, "xsl:choose may contain only xsl:when and xsl:otherwise."); + } + CompileSequence(branch.Nodes(), context.WithElement(branch), w); + w.Close(); + first = false; + } + break; + } + case "for-each": + { + if (element.Elements(Xsl + "sort").Any()) + throw Error(element, "xsl:sort is not supported."); + var set = NewName("set"); + var i = NewName("i"); + var n = NewName("n"); + var p = NewName("p"); + var s = NewName("s"); + w.Line("var " + set + " = " + XPathCompiler.Nodes(xpath.Compile(Attr(element, "select", true)!, context)) + ";"); + w.Open("for (var " + i + " = 0; " + i + " < " + set + ".Count; " + i + "++)"); + w.Line("var " + n + " = " + set + "[" + i + "];").Line("var " + p + " = " + i + " + 1;").Line("var " + s + " = " + set + ".Count;"); + CompileSequence(element.Nodes(), new Context(n, p, s, n, context.Scope, element), w); + w.Close(); + break; + } + case "variable": + { + var name = Attr(element, "name", true)!; + var id = NewName("v"); + var select = Attr(element, "select", false); + XType type; + if (select != null) + { + var value = xpath.Compile(select, context); + type = value.Type; + w.Line("var " + id + " = " + value.Code + ";"); + } + else if (HasContent(element)) + { + type = XType.Fragment; + w.Line("output.BeginCapture();"); + CompileSequence(element.Nodes(), context, w); + w.Line("var " + id + " = output.EndFragment();"); + } + else + { + type = XType.String; + w.Line("var " + id + " = \"\";"); + } + context.Scope.Add(name, new Variable(id, type)); + break; + } + case "attribute": + { + var name = ComputedName(element, context, false); + w.Line("output.Attribute(" + name + ", " + Content(element, context, w) + ");"); + break; + } + case "element": + { + if (element.Attribute("use-attribute-sets") != null) + throw Error(element, "use-attribute-sets is not supported."); + w.Line("output.StartElement(" + ComputedName(element, context, true) + ");"); + CompileSequence(element.Nodes(), context, w); + w.Line("output.EndElement();"); + break; + } + case "copy-of": + { + var value = xpath.Compile(Attr(element, "select", true)!, context); + var code = value.Type == XType.Number || value.Type == XType.Boolean ? XPathCompiler.Str(value) : value.Code; + w.Line("output.CopyOf(" + code + ");"); + break; + } + case "copy": + { + if (element.Attribute("use-attribute-sets") != null) + throw Error(element, "use-attribute-sets is not supported."); + var opened = NewName("opened"); + w.Line("var " + opened + " = output.StartCopy(" + context.Node + ");"); + w.Open("if (" + opened + " || " + context.Node + " is XDocument)"); + CompileSequence(element.Nodes(), context, w); + w.Line("if (" + opened + ")").Line(" output.EndElement();"); + w.Close(); + break; + } + case "comment": + w.Line("output.Comment(" + Content(element, context, w) + ");"); + break; + case "processing-instruction": + { + var target = XPathCompiler.Str(AttributeValueTemplate(Attr(element, "name", true)!, context)); + w.Line("output.ProcessingInstruction(" + target + ", " + Content(element, context, w) + ");"); + break; + } + case "message": + if ((string?)element.Attribute("terminate") == "yes") + w.Line("throw new System.InvalidOperationException(" + Content(element, context, w) + ");"); + break; + case "fallback": + break; + default: + throw Error(element, "xsl:" + element.Name.LocalName + " is not supported here."); + } + } + + private static bool DisablesEscaping(XElement element) => (string?)element.Attribute("disable-output-escaping") == "yes"; + + /// The parameters of xsl:apply-templates or xsl:call-template, by name, as C# expressions of type object. + private Dictionary WithParameters(XElement element, Context context, CodeWriter w) + { + var parameters = new Dictionary(); + foreach (var child in element.Elements()) + { + if (child.Name == Xsl + "sort") + throw Error(child, "xsl:sort is not supported."); + if (child.Name != Xsl + "with-param") + throw Error(child, "Unexpected " + child.Name.LocalName + "."); + var name = Attr(child, "name", true)!; + var select = Attr(child, "select", false); + var paramContext = context.WithElement(child); + if (select != null) + { + parameters[name] = XPathCompiler.Box(xpath.Compile(select, paramContext)); + } + else if (HasContent(child)) + { + var id = NewName("w"); + w.Line("output.BeginCapture();"); + CompileSequence(child.Nodes(), paramContext, w); + w.Line("var " + id + " = output.EndFragment();"); + parameters[name] = id; + } + else + { + parameters[name] = "(object)\"\""; + } + } + return parameters; + } + + /// The content of an instruction as a string (the value of an attribute, a comment...). + private string Content(XElement element, Context context, CodeWriter w) + { + var nodes = element.Nodes().Where(n => !(n is XText t && t.Value.Trim().Length == 0)).ToList(); + switch (nodes.Count) + { + case 0: + return "\"\""; + case 1 when nodes[0] is XText text: + return XPathCompiler.Literal(text.Value); + case 1 when nodes[0] is XElement e && e.Name == Xsl + "text": + return XPathCompiler.Literal(e.Value); + case 1 when nodes[0] is XElement e && e.Name == Xsl + "value-of": + return XPathCompiler.Str(xpath.Compile(Attr(e, "select", true)!, context.WithElement(e))); + } + var id = NewName("text"); + w.Line("output.BeginCapture();"); + CompileSequence(element.Nodes(), context, w); + w.Line("var " + id + " = output.EndString();"); + return id; + } + + /// The name of xsl:element or xsl:attribute; the default namespace applies to elements only. + private string ComputedName(XElement element, Context context, bool isElement) + { + var name = Attr(element, "name", true)!; + var ns = (string?)element.Attribute("namespace"); + if (ns != null && ns.IndexOf('{') >= 0) + throw Error(element, "A computed namespace is not supported."); + if (name.IndexOf('{') < 0) + { + var colon = name.IndexOf(':'); + var local = colon < 0 ? name : name.Substring(colon + 1); + if (ns == null) + ns = colon < 0 ? (isElement ? element.GetDefaultNamespace().NamespaceName : "") : Namespace(name.Substring(0, colon), element); + return Name(XName.Get(local, ns)); + } + + var value = XPathCompiler.Str(AttributeValueTemplate(name, context)); + if (ns != null) + return "XName.Get(" + value + ", " + XPathCompiler.Literal(ns) + ")"; + var table = NewName("Prefixes"); + var declarations = element.AncestorsAndSelf().SelectMany(e => e.Attributes()).Where(a => a.IsNamespaceDeclaration && a.Name.Namespace == XNamespace.Xmlns) + .GroupBy(a => a.Name.LocalName).Select(g => g.First()); + staticFields.Line("private static readonly string[] " + table + " = new[] { " + + string.Join(", ", declarations.SelectMany(a => new[] { XPathCompiler.Literal(a.Name.LocalName), XPathCompiler.Literal(a.Value) })) + " };"); + var defaultNamespace = isElement ? element.GetDefaultNamespace().NamespaceName : ""; + return "ResolveName(" + value + ", " + XPathCompiler.Literal(defaultNamespace) + ", " + table + ")"; + } + + private Compiled AttributeValueTemplate(string template, Context context) + { + var parts = new List(); + var literal = new StringBuilder(); + var i = 0; + while (i < template.Length) + { + var c = template[i]; + switch (c) + { + case '{' when i + 1 < template.Length && template[i + 1] == '{': + case '}' when i + 1 < template.Length && template[i + 1] == '}': + literal.Append(c); + i += 2; + break; + case '{': + { + var end = i + 1; + var quote = '\0'; + while (end < template.Length && (quote != '\0' || template[end] != '}')) + { + if (quote == '\0' && (template[end] == '\'' || template[end] == '"')) + quote = template[end]; + else if (template[end] == quote) + quote = '\0'; + end++; + } + if (end >= template.Length) + throw new XsltException("Unterminated expression in the attribute value template '" + template + "'."); + if (literal.Length > 0) + { + parts.Add(XPathCompiler.Literal(literal.ToString())); + literal.Clear(); + } + parts.Add(XPathCompiler.Str(xpath.Compile(template.Substring(i + 1, end - i - 1), context))); + i = end + 1; + break; + } + default: + literal.Append(c); + i++; + break; + } + } + if (literal.Length > 0 || parts.Count == 0) + parts.Add(XPathCompiler.Literal(literal.ToString())); + return new Compiled(parts.Count == 1 ? parts[0] : "string.Concat(" + string.Join(", ", parts) + ")", XType.String); + } + } +} diff --git a/src/SharpEquations.XsltGenerator/XPathCompiler.cs b/src/SharpEquations.XsltGenerator/XPathCompiler.cs new file mode 100644 index 0000000..847134b --- /dev/null +++ b/src/SharpEquations.XsltGenerator/XPathCompiler.cs @@ -0,0 +1,498 @@ +namespace SharpEquations.XsltGenerator +{ + using System.Collections.Generic; + using System.Globalization; + using System.Linq; + using System.Text; + using System.Xml.Linq; + + /// The static type of an XPath value in the generated code. + internal enum XType + { + String, + Number, + Boolean, + NodeSet, + Fragment, + Any, + } + + internal sealed class Compiled + { + public Compiled(string code, XType type) + { + Code = code; + Type = type; + } + + public string Code { get; } + + public XType Type { get; } + } + + internal sealed class Variable + { + public Variable(string id, XType type) + { + Id = id; + Type = type; + } + + public string Id { get; } + + public XType Type { get; } + } + + internal sealed class Scope + { + private readonly Dictionary variables = new Dictionary(); + private readonly Scope? parent; + + public Scope(Scope? parent) => this.parent = parent; + + public void Add(string name, Variable variable) => variables[name] = variable; + + public Variable? Find(string name) => + variables.TryGetValue(name, out var variable) ? variable : parent?.Find(name); + } + + /// + /// Where an expression is evaluated: the C# names of the context node, position and size, and of the current + /// node of XSLT; the variables in scope; the element whose namespace declarations resolve its prefixes. + /// + internal sealed class Context + { + public Context(string node, string position, string size, string current, Scope scope, XElement element) + { + Node = node; + Position = position; + Size = size; + Current = current; + Scope = scope; + Element = element; + } + + public string Node { get; } + + public string Position { get; } + + public string Size { get; } + + public string Current { get; } + + public Scope Scope { get; } + + public XElement Element { get; } + + public Context WithFocus(string node, string position, string size) => new Context(node, position, size, Current, Scope, Element); + + public Context WithScope(Scope scope) => new Context(Node, Position, Size, Current, scope, Element); + + public Context WithElement(XElement element) => new Context(Node, Position, Size, Current, Scope, element); + } + + /// XPath 1.0 expressions compiled to C# expressions over XPathFunctions. + internal sealed class XPathCompiler + { + private readonly StylesheetCompiler stylesheet; + + public XPathCompiler(StylesheetCompiler stylesheet) => this.stylesheet = stylesheet; + + public Compiled Compile(string source, Context context) => Compile(XPathParser.Parse(source), context); + + public Compiled Compile(Expr expr, Context context) + { + switch (expr) + { + case LiteralExpr literal: + return new Compiled(Literal(literal.Value), XType.String); + case NumberExpr number: + return new Compiled(Number(number.Value), XType.Number); + case VariableExpr variable: + { + var found = context.Scope.Find(variable.Name) ?? stylesheet.Global(variable.Name) + ?? throw new XsltException("The variable $" + variable.Name + " is not declared."); + return new Compiled(found.Id, found.Type); + } + case NegateExpr negate: + return new Compiled("(-" + Num(Compile(negate.Operand, context)) + ")", XType.Number); + case BinaryExpr binary: + return CompileBinary(binary, context); + case FunctionExpr function: + return CompileFunction(function, context); + case FilterExpr filter: + { + var code = Nodes(Compile(filter.Primary, context)); + foreach (var predicate in filter.Predicates) + code = "Filter(" + code + ", " + Predicate(predicate, context) + ")"; + return new Compiled(code, XType.NodeSet); + } + case PathExpr path: + return CompilePath(path, context); + default: + throw new XsltException("Unknown expression."); + } + } + + private Compiled CompileBinary(BinaryExpr binary, Context context) + { + var left = Compile(binary.Left, context); + var right = Compile(binary.Right, context); + switch (binary.Op) + { + case "or": + return new Compiled("(" + Bool(left) + " || " + Bool(right) + ")", XType.Boolean); + case "and": + return new Compiled("(" + Bool(left) + " && " + Bool(right) + ")", XType.Boolean); + case "+": + case "-": + case "*": + return new Compiled("(" + Num(left) + " " + binary.Op + " " + Num(right) + ")", XType.Number); + case "div": + return new Compiled("(" + Num(left) + " / " + Num(right) + ")", XType.Number); + case "mod": + return new Compiled("(" + Num(left) + " % " + Num(right) + ")", XType.Number); + case "|": + return new Compiled("Union(" + Nodes(left) + ", " + Nodes(right) + ")", XType.NodeSet); + default: + return CompileComparison(binary.Op, left, right); + } + } + + private static bool IsNodes(XType type) => type == XType.NodeSet || type == XType.Fragment || type == XType.Any; + + private static Compiled CompileComparison(string op, Compiled left, Compiled right) + { + if (IsNodes(left.Type) || IsNodes(right.Type)) + return new Compiled("Compare(" + Box(left) + ", " + Box(right) + ", Comparison." + ComparisonName(op) + ")", XType.Boolean); + var csharp = op == "=" ? "==" : op; + string code; + if (op != "=" && op != "!=") + code = "(" + Num(left) + " " + csharp + " " + Num(right) + ")"; + else if (left.Type == XType.Boolean || right.Type == XType.Boolean) + code = "(" + Bool(left) + " " + csharp + " " + Bool(right) + ")"; + else if (left.Type == XType.Number || right.Type == XType.Number) + code = "(" + Num(left) + " " + csharp + " " + Num(right) + ")"; + else + code = "(" + Str(left) + " " + csharp + " " + Str(right) + ")"; + return new Compiled(code, XType.Boolean); + } + + private static string ComparisonName(string op) + { + switch (op) + { + case "=": + return "Equal"; + case "!=": + return "NotEqual"; + case "<": + return "Less"; + case "<=": + return "LessOrEqual"; + case ">": + return "Greater"; + default: + return "GreaterOrEqual"; + } + } + + private Compiled CompileFunction(FunctionExpr function, Context context) + { + var args = function.Arguments.Select(a => Compile(a, context)).ToList(); + void Arity(int min, int max) + { + if (args.Count < min || args.Count > max) + throw new XsltException("Wrong number of arguments to " + function.Name + "()."); + } + + string StringArgument(int i) => args.Count > i ? Str(args[i]) : "StringValue(" + context.Node + ")"; + + switch (function.Name) + { + case "last": + Arity(0, 0); + return new Compiled("((double)" + context.Size + ")", XType.Number); + case "position": + Arity(0, 0); + return new Compiled("((double)" + context.Position + ")", XType.Number); + case "count": + Arity(1, 1); + return new Compiled("Count(" + Nodes(args[0]) + ")", XType.Number); + case "sum": + Arity(1, 1); + return new Compiled("Sum(" + Nodes(args[0]) + ")", XType.Number); + case "local-name": + case "namespace-uri": + case "name": + { + Arity(0, 1); + var method = function.Name == "local-name" ? "LocalName" : function.Name == "name" ? "Name" : "NamespaceUri"; + return new Compiled(method + "(" + (args.Count == 0 ? context.Node : Nodes(args[0])) + ")", XType.String); + } + case "string": + Arity(0, 1); + return new Compiled(StringArgument(0), XType.String); + case "concat": + if (args.Count < 2) + throw new XsltException("concat() takes at least two arguments."); + return new Compiled("string.Concat(" + string.Join(", ", args.Select(Str)) + ")", XType.String); + case "starts-with": + Arity(2, 2); + return new Compiled("StartsWith(" + Str(args[0]) + ", " + Str(args[1]) + ")", XType.Boolean); + case "contains": + Arity(2, 2); + return new Compiled("Contains(" + Str(args[0]) + ", " + Str(args[1]) + ")", XType.Boolean); + case "substring-before": + Arity(2, 2); + return new Compiled("SubstringBefore(" + Str(args[0]) + ", " + Str(args[1]) + ")", XType.String); + case "substring-after": + Arity(2, 2); + return new Compiled("SubstringAfter(" + Str(args[0]) + ", " + Str(args[1]) + ")", XType.String); + case "substring": + Arity(2, 3); + return new Compiled("Substring(" + string.Join(", ", new[] { Str(args[0]) }.Concat(args.Skip(1).Select(Num))) + ")", XType.String); + case "string-length": + Arity(0, 1); + return new Compiled("StringLength(" + StringArgument(0) + ")", XType.Number); + case "normalize-space": + Arity(0, 1); + return new Compiled("NormalizeSpace(" + StringArgument(0) + ")", XType.String); + case "translate": + Arity(3, 3); + return new Compiled("Translate(" + Str(args[0]) + ", " + Str(args[1]) + ", " + Str(args[2]) + ")", XType.String); + case "boolean": + Arity(1, 1); + return new Compiled(Bool(args[0]), XType.Boolean); + case "not": + Arity(1, 1); + return new Compiled("(!" + Bool(args[0]) + ")", XType.Boolean); + case "true": + Arity(0, 0); + return new Compiled("true", XType.Boolean); + case "false": + Arity(0, 0); + return new Compiled("false", XType.Boolean); + case "number": + Arity(0, 1); + return new Compiled(args.Count == 0 ? "Num(StringValue(" + context.Node + "))" : Num(args[0]), XType.Number); + case "floor": + Arity(1, 1); + return new Compiled("System.Math.Floor(" + Num(args[0]) + ")", XType.Number); + case "ceiling": + Arity(1, 1); + return new Compiled("System.Math.Ceiling(" + Num(args[0]) + ")", XType.Number); + case "round": + Arity(1, 1); + return new Compiled("Round(" + Num(args[0]) + ")", XType.Number); + case "current": + Arity(0, 0); + return new Compiled("Single(" + context.Current + ")", XType.NodeSet); + default: + throw new XsltException("The function " + function.Name + "() is not supported."); + } + } + + private Compiled CompilePath(PathExpr path, Context context) + { + string code; + if (path.Start != null) + code = Nodes(Compile(path.Start, context)); + else if (path.Absolute) + code = "Root(" + context.Node + ")"; + else + code = "Single(" + context.Node + ")"; + + var steps = path.Steps; + for (var i = 0; i < steps.Count; i++) + { + var step = steps[i]; + if (step.Axis == "self" && step.Test.Kind == TestKind.Node && step.Predicates.Count == 0) + continue; + + // //name: descendant::name, when the predicates do not count positions among the children + if (step.IsDescendantOrSelfNode && i + 1 < steps.Count && steps[i + 1].Axis == "child" + && steps[i + 1].Predicates.All(p => !IsPositional(p, context))) + { + var next = steps[i + 1]; + code = StepCode(code, "Descendant", next.Test, next.Predicates, false, context); + i++; + continue; + } + code = StepCode(code, AxisName(step.Axis), step.Test, step.Predicates, step.Axis == "attribute", context); + } + return new Compiled(code, XType.NodeSet); + } + + private string StepCode(string input, string axis, NodeTestSyntax test, List predicates, bool attributeAxis, Context context) + { + var testField = stylesheet.NodeTest(test, context.Element); + if (predicates.Count == 0) + return "Step(" + input + ", Axis." + axis + ", " + testField + ")"; + return "Step(" + input + ", Axis." + axis + ", " + testField + ", " + string.Join(", ", predicates.Select(p => Predicate(p, context))) + ")"; + } + + public static string AxisName(string axis) + { + switch (axis) + { + case "child": + return "Child"; + case "descendant": + return "Descendant"; + case "descendant-or-self": + return "DescendantOrSelf"; + case "parent": + return "Parent"; + case "ancestor": + return "Ancestor"; + case "ancestor-or-self": + return "AncestorOrSelf"; + case "following-sibling": + return "FollowingSibling"; + case "preceding-sibling": + return "PrecedingSibling"; + case "self": + return "Self"; + case "attribute": + return "Attribute"; + default: + throw new XsltException("The axis " + axis + " is not supported."); + } + } + + /// A lambda of XPathPredicate: the predicate evaluated with the node, its position and the size. + public string Predicate(Expr predicate, Context context) + { + var n = stylesheet.NewName("n"); + var p = stylesheet.NewName("p"); + var s = stylesheet.NewName("s"); + var value = Compile(predicate, context.WithFocus(n, p, s)); + string body; + switch (value.Type) + { + case XType.Number: + body = p + " == " + value.Code; + break; + case XType.Any: + body = "Predicate(" + value.Code + ", " + p + ")"; + break; + default: + body = Bool(value); + break; + } + return "(" + n + ", " + p + ", " + s + ") => " + body; + } + + /// + /// Whether the value of a predicate depends on the position of the node among the others: a number (or a + /// value that may be one), or an expression that reads position() or last(). + /// + public bool IsPositional(Expr predicate, Context context) + { + if (UsesFocus(predicate)) + return true; + var type = Compile(predicate, context.WithFocus("n", "p", "s")).Type; + return type == XType.Number || type == XType.Any; + } + + // whether the expression reads the position or size of its context (not that of a nested predicate) + private static bool UsesFocus(Expr expr) + { + switch (expr) + { + case FunctionExpr function when function.Name == "position" || function.Name == "last": + return true; + case FunctionExpr function: + return function.Arguments.Any(UsesFocus); + case BinaryExpr binary: + return UsesFocus(binary.Left) || UsesFocus(binary.Right); + case NegateExpr negate: + return UsesFocus(negate.Operand); + case FilterExpr filter: + return UsesFocus(filter.Primary); + case PathExpr path: + return path.Start != null && UsesFocus(path.Start); + default: + return false; + } + } + + // conversions + + public static string Str(Compiled value) => value.Type == XType.String ? value.Code : "Str(" + value.Code + ")"; + + public static string Num(Compiled value) => value.Type == XType.Number ? value.Code : "Num(" + value.Code + ")"; + + public static string Bool(Compiled value) => value.Type == XType.Boolean ? value.Code : "Bool(" + value.Code + ")"; + + public static string Box(Compiled value) => "(object)" + value.Code; + + public static string Nodes(Compiled value) + { + switch (value.Type) + { + case XType.NodeSet: + return value.Code; + case XType.Any: + return "Nodes(" + value.Code + ")"; + case XType.Fragment: + throw new XsltException("A result tree fragment cannot be used as a node-set in XSLT 1.0."); + default: + throw new XsltException("A " + value.Type + " cannot be used as a node-set."); + } + } + + public static string TypeName(XType type) + { + switch (type) + { + case XType.String: + return "string"; + case XType.Number: + return "double"; + case XType.Boolean: + return "bool"; + case XType.NodeSet: + return "List"; + case XType.Fragment: + return "ResultFragment"; + default: + return "object"; + } + } + + public static string Literal(string value) + { + var result = new StringBuilder("\""); + foreach (var c in value) + { + switch (c) + { + case '\\': + result.Append("\\\\"); + break; + case '"': + result.Append("\\\""); + break; + case var _ when c < 0x20 || c >= 0x7F: + result.Append("\\u").Append(((int)c).ToString("X4", CultureInfo.InvariantCulture)); + break; + default: + result.Append(c); + break; + } + } + return result.Append('"').ToString(); + } + + public static string Number(double value) + { + if (double.IsNaN(value)) + return "double.NaN"; + if (double.IsPositiveInfinity(value)) + return "double.PositiveInfinity"; + return value.ToString("R", CultureInfo.InvariantCulture) + "D"; + } + } +} diff --git a/src/SharpEquations.XsltGenerator/XPathParser.cs b/src/SharpEquations.XsltGenerator/XPathParser.cs new file mode 100644 index 0000000..39e7a6c --- /dev/null +++ b/src/SharpEquations.XsltGenerator/XPathParser.cs @@ -0,0 +1,728 @@ +namespace SharpEquations.XsltGenerator +{ + using System.Collections.Generic; + using System.Globalization; + using System.Text; + + internal abstract class Expr + { + } + + internal sealed class BinaryExpr : Expr + { + public BinaryExpr(string op, Expr left, Expr right) + { + Op = op; + Left = left; + Right = right; + } + + public string Op { get; } + + public Expr Left { get; } + + public Expr Right { get; } + } + + internal sealed class NegateExpr : Expr + { + public NegateExpr(Expr operand) => Operand = operand; + + public Expr Operand { get; } + } + + internal sealed class LiteralExpr : Expr + { + public LiteralExpr(string value) => Value = value; + + public string Value { get; } + } + + internal sealed class NumberExpr : Expr + { + public NumberExpr(double value) => Value = value; + + public double Value { get; } + } + + internal sealed class VariableExpr : Expr + { + public VariableExpr(string name) => Name = name; + + public string Name { get; } + } + + internal sealed class FunctionExpr : Expr + { + public FunctionExpr(string name, List arguments) + { + Name = name; + Arguments = arguments; + } + + public string Name { get; } + + public List Arguments { get; } + } + + internal sealed class FilterExpr : Expr + { + public FilterExpr(Expr primary, List predicates) + { + Primary = primary; + Predicates = predicates; + } + + public Expr Primary { get; } + + public List Predicates { get; } + } + + /// A location path (no start: relative to the context node, or absolute), or a filter expression followed by steps. + internal sealed class PathExpr : Expr + { + public PathExpr(Expr? start, bool absolute, List steps) + { + Start = start; + Absolute = absolute; + Steps = steps; + } + + public Expr? Start { get; } + + public bool Absolute { get; } + + public List Steps { get; } + } + + internal enum TestKind + { + Name, + Namespace, + AnyName, + Node, + Text, + Comment, + ProcessingInstruction, + } + + internal sealed class NodeTestSyntax + { + public NodeTestSyntax(TestKind kind, string? prefix = null, string? local = null) + { + Kind = kind; + Prefix = prefix; + Local = local; + } + + public TestKind Kind { get; } + + public string? Prefix { get; } + + /// The local name, or the target of a processing-instruction() test. + public string? Local { get; } + } + + internal sealed class Step + { + public Step(string axis, NodeTestSyntax test, List predicates) + { + Axis = axis; + Test = test; + Predicates = predicates; + } + + public string Axis { get; } + + public NodeTestSyntax Test { get; } + + public List Predicates { get; } + + public bool IsDescendantOrSelfNode => Axis == "descendant-or-self" && Test.Kind == TestKind.Node && Predicates.Count == 0; + } + + /// A parser of XPath 1.0 expressions. + internal sealed class XPathParser + { + private enum TokenKind + { + Name, + NameStar, + Star, + Literal, + Number, + Variable, + Operator, + LeftParen, + RightParen, + LeftBracket, + RightBracket, + Dot, + DotDot, + At, + Comma, + ColonColon, + Slash, + DoubleSlash, + Pipe, + End, + } + + private sealed class Token + { + public Token(TokenKind kind, string text) + { + Kind = kind; + Text = text; + } + + public TokenKind Kind { get; } + + public string Text { get; } + } + + private static readonly HashSet NodeTypes = new HashSet { "node", "text", "comment", "processing-instruction" }; + + private static readonly HashSet Axes = new HashSet + { + "ancestor", "ancestor-or-self", "attribute", "child", "descendant", "descendant-or-self", "following", + "following-sibling", "namespace", "parent", "preceding", "preceding-sibling", "self", + }; + + private readonly string source; + private readonly List tokens; + private int index; + + private XPathParser(string source) + { + this.source = source; + tokens = Tokenize(source); + } + + public static Expr Parse(string source) + { + var parser = new XPathParser(source); + var expr = parser.ParseOr(); + parser.Expect(TokenKind.End); + return expr; + } + + private Token Current => tokens[index]; + + private Token Peek(int offset) => tokens[System.Math.Min(index + offset, tokens.Count - 1)]; + + private XsltException Error(string message) => new XsltException(message + " in the expression '" + source + "'."); + + private Token Expect(TokenKind kind) + { + if (Current.Kind != kind) + throw Error("Expected " + kind + ", found '" + Current.Text + "'"); + return tokens[index++]; + } + + private bool IsOperator(string op) => Current.Kind == TokenKind.Operator && Current.Text == op; + + private Expr ParseOr() + { + var left = ParseAnd(); + while (IsOperator("or")) + { + index++; + left = new BinaryExpr("or", left, ParseAnd()); + } + return left; + } + + private Expr ParseAnd() + { + var left = ParseEquality(); + while (IsOperator("and")) + { + index++; + left = new BinaryExpr("and", left, ParseEquality()); + } + return left; + } + + private Expr ParseEquality() + { + var left = ParseRelational(); + while (IsOperator("=") || IsOperator("!=")) + { + var op = tokens[index++].Text; + left = new BinaryExpr(op, left, ParseRelational()); + } + return left; + } + + private Expr ParseRelational() + { + var left = ParseAdditive(); + while (IsOperator("<") || IsOperator("<=") || IsOperator(">") || IsOperator(">=")) + { + var op = tokens[index++].Text; + left = new BinaryExpr(op, left, ParseAdditive()); + } + return left; + } + + private Expr ParseAdditive() + { + var left = ParseMultiplicative(); + while (IsOperator("+") || IsOperator("-")) + { + var op = tokens[index++].Text; + left = new BinaryExpr(op, left, ParseMultiplicative()); + } + return left; + } + + private Expr ParseMultiplicative() + { + var left = ParseUnary(); + while (IsOperator("*") || IsOperator("div") || IsOperator("mod")) + { + var op = tokens[index++].Text; + left = new BinaryExpr(op, left, ParseUnary()); + } + return left; + } + + private Expr ParseUnary() + { + if (IsOperator("-")) + { + index++; + return new NegateExpr(ParseUnary()); + } + return ParseUnion(); + } + + private Expr ParseUnion() + { + var left = ParsePath(); + while (Current.Kind == TokenKind.Pipe) + { + index++; + left = new BinaryExpr("|", left, ParsePath()); + } + return left; + } + + private bool StartsFilter() + { + switch (Current.Kind) + { + case TokenKind.Variable: + case TokenKind.LeftParen: + case TokenKind.Literal: + case TokenKind.Number: + return true; + case TokenKind.Name: + return Peek(1).Kind == TokenKind.LeftParen && !NodeTypes.Contains(Current.Text); + default: + return false; + } + } + + private bool StartsStep() + { + switch (Current.Kind) + { + case TokenKind.Name: + case TokenKind.NameStar: + case TokenKind.Star: + case TokenKind.Dot: + case TokenKind.DotDot: + case TokenKind.At: + return true; + default: + return false; + } + } + + private Expr ParsePath() + { + if (StartsFilter()) + { + var primary = ParsePrimary(); + var predicates = ParsePredicates(); + Expr filter = predicates.Count == 0 ? primary : new FilterExpr(primary, predicates); + if (Current.Kind != TokenKind.Slash && Current.Kind != TokenKind.DoubleSlash) + return filter; + var steps = new List(); + ParseRelativePath(steps); + return new PathExpr(filter, false, steps); + } + + switch (Current.Kind) + { + case TokenKind.Slash: + { + index++; + var steps = new List(); + if (StartsStep()) + { + steps.Add(ParseStep()); + ParseRelativePath(steps); + } + return new PathExpr(null, true, steps); + } + case TokenKind.DoubleSlash: + { + var steps = new List(); + ParseRelativePath(steps); + return new PathExpr(null, true, steps); + } + default: + { + var steps = new List { ParseStep() }; + ParseRelativePath(steps); + return new PathExpr(null, false, steps); + } + } + } + + // the steps after the current one: ('/' Step | '//' Step)* + private void ParseRelativePath(List steps) + { + while (true) + { + switch (Current.Kind) + { + case TokenKind.Slash: + index++; + steps.Add(ParseStep()); + break; + case TokenKind.DoubleSlash: + index++; + steps.Add(new Step("descendant-or-self", new NodeTestSyntax(TestKind.Node), new List())); + steps.Add(ParseStep()); + break; + default: + return; + } + } + } + + private Step ParseStep() + { + switch (Current.Kind) + { + case TokenKind.Dot: + index++; + return new Step("self", new NodeTestSyntax(TestKind.Node), new List()); + case TokenKind.DotDot: + index++; + return new Step("parent", new NodeTestSyntax(TestKind.Node), new List()); + } + + var axis = "child"; + if (Current.Kind == TokenKind.At) + { + index++; + axis = "attribute"; + } + else if (Current.Kind == TokenKind.Name && Peek(1).Kind == TokenKind.ColonColon) + { + axis = Current.Text; + if (!Axes.Contains(axis)) + throw Error("Unknown axis '" + axis + "'"); + index += 2; + } + var test = ParseNodeTest(); + return new Step(axis, test, ParsePredicates()); + } + + private NodeTestSyntax ParseNodeTest() + { + var token = Current; + switch (token.Kind) + { + case TokenKind.Star: + index++; + return new NodeTestSyntax(TestKind.AnyName); + case TokenKind.NameStar: + index++; + return new NodeTestSyntax(TestKind.Namespace, token.Text.Substring(0, token.Text.Length - 2)); + case TokenKind.Name when Peek(1).Kind == TokenKind.LeftParen && NodeTypes.Contains(token.Text): + { + index += 2; + string? target = null; + if (token.Text == "processing-instruction" && Current.Kind == TokenKind.Literal) + target = tokens[index++].Text; + Expect(TokenKind.RightParen); + switch (token.Text) + { + case "node": + return new NodeTestSyntax(TestKind.Node); + case "text": + return new NodeTestSyntax(TestKind.Text); + case "comment": + return new NodeTestSyntax(TestKind.Comment); + default: + return new NodeTestSyntax(TestKind.ProcessingInstruction, local: target); + } + } + case TokenKind.Name: + { + index++; + var colon = token.Text.IndexOf(':'); + return colon < 0 + ? new NodeTestSyntax(TestKind.Name, null, token.Text) + : new NodeTestSyntax(TestKind.Name, token.Text.Substring(0, colon), token.Text.Substring(colon + 1)); + } + default: + throw Error("Expected a node test, found '" + token.Text + "'"); + } + } + + private List ParsePredicates() + { + var predicates = new List(); + while (Current.Kind == TokenKind.LeftBracket) + { + index++; + predicates.Add(ParseOr()); + Expect(TokenKind.RightBracket); + } + return predicates; + } + + private Expr ParsePrimary() + { + var token = tokens[index++]; + switch (token.Kind) + { + case TokenKind.Variable: + return new VariableExpr(token.Text); + case TokenKind.Literal: + return new LiteralExpr(token.Text); + case TokenKind.Number: + return new NumberExpr(double.Parse(token.Text, NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture)); + case TokenKind.LeftParen: + { + var expr = ParseOr(); + Expect(TokenKind.RightParen); + return expr; + } + default: + { + Expect(TokenKind.LeftParen); + var arguments = new List(); + if (Current.Kind != TokenKind.RightParen) + { + arguments.Add(ParseOr()); + while (Current.Kind == TokenKind.Comma) + { + index++; + arguments.Add(ParseOr()); + } + } + Expect(TokenKind.RightParen); + return new FunctionExpr(token.Text, arguments); + } + } + } + + private static bool IsNameStart(char c) => char.IsLetter(c) || c == '_'; + + private static bool IsNameChar(char c) => char.IsLetterOrDigit(c) || c == '_' || c == '-' || c == '.' || char.GetUnicodeCategory(c) == UnicodeCategory.NonSpacingMark; + + private static string ReadName(string s, ref int i) + { + var start = i; + while (i < s.Length && IsNameChar(s[i])) + i++; + return s.Substring(start, i - start); + } + + private static List Tokenize(string s) + { + var tokens = new List(); + var i = 0; + while (true) + { + while (i < s.Length && (s[i] == ' ' || s[i] == '\t' || s[i] == '\r' || s[i] == '\n')) + i++; + if (i >= s.Length) + { + tokens.Add(new Token(TokenKind.End, "end")); + return tokens; + } + + // after a token that ends an operand, * and the operator names are operators (XPath 1.0, 3.7) + var operand = tokens.Count > 0 && EndsOperand(tokens[tokens.Count - 1]); + var c = s[i]; + var next = i + 1 < s.Length ? s[i + 1] : '\0'; + switch (c) + { + case '(': + tokens.Add(new Token(TokenKind.LeftParen, "(")); + i++; + break; + case ')': + tokens.Add(new Token(TokenKind.RightParen, ")")); + i++; + break; + case '[': + tokens.Add(new Token(TokenKind.LeftBracket, "[")); + i++; + break; + case ']': + tokens.Add(new Token(TokenKind.RightBracket, "]")); + i++; + break; + case ',': + tokens.Add(new Token(TokenKind.Comma, ",")); + i++; + break; + case '@': + tokens.Add(new Token(TokenKind.At, "@")); + i++; + break; + case '|': + tokens.Add(new Token(TokenKind.Pipe, "|")); + i++; + break; + case ':' when next == ':': + tokens.Add(new Token(TokenKind.ColonColon, "::")); + i += 2; + break; + case '/' when next == '/': + tokens.Add(new Token(TokenKind.DoubleSlash, "//")); + i += 2; + break; + case '/': + tokens.Add(new Token(TokenKind.Slash, "/")); + i++; + break; + case '.' when next == '.': + tokens.Add(new Token(TokenKind.DotDot, "..")); + i += 2; + break; + case '.' when !char.IsDigit(next): + tokens.Add(new Token(TokenKind.Dot, ".")); + i++; + break; + case '*': + tokens.Add(operand ? new Token(TokenKind.Operator, "*") : new Token(TokenKind.Star, "*")); + i++; + break; + case '+': + case '-': + case '=': + tokens.Add(new Token(TokenKind.Operator, c.ToString())); + i++; + break; + case '!' when next == '=': + tokens.Add(new Token(TokenKind.Operator, "!=")); + i += 2; + break; + case '<': + case '>': + if (next == '=') + { + tokens.Add(new Token(TokenKind.Operator, c + "=")); + i += 2; + } + else + { + tokens.Add(new Token(TokenKind.Operator, c.ToString())); + i++; + } + break; + case '"': + case '\'': + { + var end = s.IndexOf(c, i + 1); + if (end < 0) + throw new XsltException("Unterminated literal in the expression '" + s + "'."); + tokens.Add(new Token(TokenKind.Literal, s.Substring(i + 1, end - i - 1))); + i = end + 1; + break; + } + case '$': + { + i++; + var name = ReadName(s, ref i); + if (i < s.Length && s[i] == ':' && i + 1 < s.Length && IsNameStart(s[i + 1])) + { + i++; + name += ":" + ReadName(s, ref i); + } + tokens.Add(new Token(TokenKind.Variable, name)); + break; + } + default: + if (char.IsDigit(c) || c == '.') + { + var number = new StringBuilder(); + while (i < s.Length && (char.IsDigit(s[i]) || s[i] == '.')) + number.Append(s[i++]); + tokens.Add(new Token(TokenKind.Number, number.ToString())); + } + else if (IsNameStart(c)) + { + var name = ReadName(s, ref i); + if (operand && (name == "and" || name == "or" || name == "div" || name == "mod")) + { + tokens.Add(new Token(TokenKind.Operator, name)); + } + else if (i + 1 < s.Length && s[i] == ':' && s[i + 1] == '*') + { + tokens.Add(new Token(TokenKind.NameStar, name + ":*")); + i += 2; + } + else if (i + 1 < s.Length && s[i] == ':' && IsNameStart(s[i + 1])) + { + i++; + tokens.Add(new Token(TokenKind.Name, name + ":" + ReadName(s, ref i))); + } + else + { + tokens.Add(new Token(TokenKind.Name, name)); + } + } + else + { + throw new XsltException("Unexpected character '" + c + "' in the expression '" + s + "'."); + } + break; + } + } + } + + private static bool EndsOperand(Token token) + { + switch (token.Kind) + { + case TokenKind.At: + case TokenKind.ColonColon: + case TokenKind.LeftParen: + case TokenKind.LeftBracket: + case TokenKind.Comma: + case TokenKind.Operator: + case TokenKind.Slash: + case TokenKind.DoubleSlash: + case TokenKind.Pipe: + return false; + default: + return true; + } + } + } + + internal sealed class XsltException : System.Exception + { + public XsltException(string message) + : base(message) + { + } + + /// The line in the stylesheet, from 1, or 0 if unknown. + public int Line { get; set; } + + public int Column { get; set; } + } +} diff --git a/src/SharpEquations.XsltGenerator/XsltGenerator.cs b/src/SharpEquations.XsltGenerator/XsltGenerator.cs new file mode 100644 index 0000000..1363b85 --- /dev/null +++ b/src/SharpEquations.XsltGenerator/XsltGenerator.cs @@ -0,0 +1,73 @@ +namespace SharpEquations.XsltGenerator +{ + using System; + using System.IO; + using System.Linq; + using System.Text; + using System.Xml; + using System.Xml.Linq; + using Microsoft.CodeAnalysis; + using Microsoft.CodeAnalysis.Text; + + /// + /// Each XSLT 1.0 stylesheet among the additional files (*.xsl) compiled to a C# class of + /// SharpEquations.Internal.Xslt, named after the file (omml2mml.xsl: Omml2mmlStylesheet), whose + /// Transform(XElement) runs it over the runtime of that namespace. + /// + [Generator(LanguageNames.CSharp)] + public sealed class XsltGenerator : IIncrementalGenerator + { + private static readonly DiagnosticDescriptor CannotCompile = new DiagnosticDescriptor( + "XSLT001", "The stylesheet cannot be compiled", "{0}", "SharpEquations.Xslt", DiagnosticSeverity.Error, isEnabledByDefault: true); + + public void Initialize(IncrementalGeneratorInitializationContext context) + { + var stylesheets = context.AdditionalTextsProvider + .Where(static file => file.Path.EndsWith(".xsl", StringComparison.OrdinalIgnoreCase)) + .Select(static (file, cancellationToken) => (file.Path, Text: file.GetText(cancellationToken)?.ToString())); + context.RegisterSourceOutput(stylesheets, static (output, file) => Generate(output, file.Path, file.Text)); + } + + public static string ClassName(string path) + { + var name = new StringBuilder(); + foreach (var c in Path.GetFileNameWithoutExtension(path)) + { + if (char.IsLetterOrDigit(c) || c == '_') + name.Append(name.Length == 0 ? char.ToUpperInvariant(c) : c); + } + return name + "Stylesheet"; + } + + private static void Generate(SourceProductionContext output, string path, string? text) + { + if (text == null) + return; + var fileName = Path.GetFileName(path); + try + { + var settings = new XmlReaderSettings { DtdProcessing = DtdProcessing.Prohibit, XmlResolver = null }; + XDocument document; + using (var reader = XmlReader.Create(new StringReader(text), settings)) + document = XDocument.Load(reader, LoadOptions.SetLineInfo | LoadOptions.PreserveWhitespace); + var className = ClassName(path); + var source = new StylesheetCompiler(document.Root!, className, fileName).Compile(); + output.AddSource(className + ".g.cs", SourceText.From(source, Encoding.UTF8)); + } + catch (XsltException e) + { + output.ReportDiagnostic(Diagnostic.Create(CannotCompile, Location(path, e.Line, e.Column), e.Message)); + } + catch (XmlException e) + { + output.ReportDiagnostic(Diagnostic.Create(CannotCompile, Location(path, e.LineNumber, e.LinePosition), e.Message)); + } + } + + private static Location Location(string path, int line, int column) + { + var position = new LinePosition(Math.Max(0, line - 1), Math.Max(0, column - 1)); + return Microsoft.CodeAnalysis.Location.Create(path, default, new LinePositionSpan(position, position)); + } + } +} diff --git a/src/SharpEquations/Internal/EquationStylesheets.cs b/src/SharpEquations/Internal/EquationStylesheets.cs index 7dc840b..4ff801c 100644 --- a/src/SharpEquations/Internal/EquationStylesheets.cs +++ b/src/SharpEquations/Internal/EquationStylesheets.cs @@ -1,50 +1,40 @@ namespace SharpEquations.Internal { using System; - using System.Collections.Concurrent; using System.Collections.Generic; - using System.IO; - using System.Xml; using System.Xml.Linq; - using System.Xml.Xsl; + using SharpEquations.Internal.Xslt; /// - /// The XSLT 1.0 stylesheets embedded under Resources/Equations, compiled once on first use. + /// The XSLT 1.0 stylesheets of Resources, compiled to C# at build time by SharpEquations.XsltGenerator + /// (Internal/Xslt), so that they run without generating code at run time. /// internal static class EquationStylesheets { - private const string ResourcePrefix = "SharpEquations.Resources."; - /// /// The deepest input accepted: the stylesheets recurse over the tree, and a stack overflow cannot be caught. /// public const int MaxDepth = 512; - private static readonly ConcurrentDictionary> Compiled = - new ConcurrentDictionary>(StringComparer.Ordinal); - - /// - /// Runs the stylesheet and parses its output, or returns null when the output is empty. - /// The output goes through text so that disable-output-escaping, used by mtef2mml.xsl - /// to write character references, takes effect. - /// + /// Runs the stylesheet, or returns null when its output is empty. /// The input is nested more than levels deep. public static XElement Transform(string name, XElement input) { if (Depth(input) > MaxDepth) throw new ArgumentException("The equation is nested too deeply.", nameof(input)); - var stylesheet = Compiled.GetOrAdd(name, n => new Lazy(() => Load(n))).Value; - var output = new StringWriter(); - using (var reader = new XDocument(input).CreateReader()) + switch (name) { - stylesheet.Transform(reader, null, output); + case "omml2mml.xsl": + return Omml2mmlStylesheet.Transform(input); + case "mml2omml.xsl": + return Mml2ommlStylesheet.Transform(input); + case "ctop.xsl": + return CtopStylesheet.Transform(input); + case "mtef2mml.xsl": + return Mtef2mmlStylesheet.Transform(input); + default: + throw new ArgumentOutOfRangeException(nameof(name), name, "No such stylesheet."); } - var text = output.ToString(); - if (text.Trim().Length == 0) - return null; - var settings = new XmlReaderSettings { DtdProcessing = DtdProcessing.Prohibit, XmlResolver = null }; - using (var reader = XmlReader.Create(new StringReader(text), settings)) - return XElement.Load(reader, LoadOptions.PreserveWhitespace); } private static int Depth(XElement root) @@ -64,21 +54,5 @@ private static int Depth(XElement root) } return depth; } - - private static XslCompiledTransform Load(string name) - { - using (var stream = typeof(EquationStylesheets).Assembly.GetManifestResourceStream(ResourcePrefix + name)) - { - if (stream == null) - throw new FileNotFoundException("Embedded stylesheet not found.", name); - var settings = new XmlReaderSettings { DtdProcessing = DtdProcessing.Prohibit, XmlResolver = null }; - using (var reader = XmlReader.Create(stream, settings)) - { - var stylesheet = new XslCompiledTransform(); - stylesheet.Load(reader, XsltSettings.Default, null); - return stylesheet; - } - } - } } } diff --git a/src/SharpEquations/Internal/Xslt/NodeTest.cs b/src/SharpEquations/Internal/Xslt/NodeTest.cs new file mode 100644 index 0000000..0b96e75 --- /dev/null +++ b/src/SharpEquations/Internal/Xslt/NodeTest.cs @@ -0,0 +1,75 @@ +#nullable enable + +namespace SharpEquations.Internal.Xslt +{ + using System.Xml.Linq; + + /// The node test of an XPath step: a name, a namespace wildcard, * or a node type. + internal sealed class NodeTest + { + private enum Kind + { + Name, + Namespace, + AnyName, + Node, + Text, + Comment, + ProcessingInstruction, + } + + private readonly Kind kind; + private readonly XName? name; + private readonly string? text; + + private NodeTest(Kind kind, XName? name = null, string? text = null) + { + this.kind = kind; + this.name = name; + this.text = text; + } + + public static NodeTest Named(XName name) => new NodeTest(Kind.Name, name); + + public static NodeTest InNamespace(string namespaceName) => new NodeTest(Kind.Namespace, text: namespaceName); + + public static readonly NodeTest AnyName = new NodeTest(Kind.AnyName); + + public static readonly NodeTest Node = new NodeTest(Kind.Node); + + public static readonly NodeTest Text = new NodeTest(Kind.Text); + + public static readonly NodeTest Comment = new NodeTest(Kind.Comment); + + public static NodeTest ProcessingInstruction(string? target) => new NodeTest(Kind.ProcessingInstruction, text: target); + + /// + /// Whether the node passes the test, on an axis whose principal node type is the attribute (the attribute + /// axis) or the element (every other axis). + /// + public bool Matches(XObject node, bool attributeAxis) + { + switch (kind) + { + case Kind.Name: + return attributeAxis + ? node is XAttribute attribute && attribute.Name == name + : node is XElement element && element.Name == name; + case Kind.Namespace: + return attributeAxis + ? node is XAttribute attribute2 && attribute2.Name.NamespaceName == text + : node is XElement element2 && element2.Name.NamespaceName == text; + case Kind.AnyName: + return attributeAxis ? node is XAttribute : node is XElement; + case Kind.Node: + return true; + case Kind.Text: + return node is XText; + case Kind.Comment: + return node is XComment; + default: + return node is XProcessingInstruction instruction && (text == null || instruction.Target == text); + } + } + } +} diff --git a/src/SharpEquations/Internal/Xslt/ResultFragment.cs b/src/SharpEquations/Internal/Xslt/ResultFragment.cs new file mode 100644 index 0000000..703040d --- /dev/null +++ b/src/SharpEquations/Internal/Xslt/ResultFragment.cs @@ -0,0 +1,43 @@ +#nullable enable + +namespace SharpEquations.Internal.Xslt +{ + using System.Xml.Linq; + + /// A result tree fragment: the value of a variable or parameter given by its content. + internal sealed class ResultFragment + { + public ResultFragment(XElement root) => Root = root; + + /// The nodes of the fragment, as the children of an element that is not part of it. + public XElement Root { get; } + + public string Value => Root.Value; + } + + /// The parameters passed by xsl:apply-templates. + internal sealed class TemplateArguments + { + private readonly string[] names; + private readonly object[] values; + + public TemplateArguments(string[] names, object[] values) + { + this.names = names; + this.values = values; + } + + public object? Get(string name) + { + for (var i = 0; i < names.Length; i++) + { + if (names[i] == name) + return values[i]; + } + return null; + } + } + + /// An XPath predicate, or any expression evaluated with a context node, position and size. + internal delegate bool XPathPredicate(XObject node, int position, int size); +} diff --git a/src/SharpEquations/Internal/Xslt/Stylesheets.cs b/src/SharpEquations/Internal/Xslt/Stylesheets.cs new file mode 100644 index 0000000..b75d53e --- /dev/null +++ b/src/SharpEquations/Internal/Xslt/Stylesheets.cs @@ -0,0 +1,33 @@ +#nullable enable + +namespace SharpEquations.Internal.Xslt +{ + using System.Xml.Linq; + + // The stylesheets of Resources, whose Transform SharpEquations.XsltGenerator implements from Resources/.xsl. + // They are declared here so that the code that uses them compiles without the generator (as Parlot compiles it). + + /// omml2mml.xsl: OMML to MathML. + internal sealed partial class Omml2mmlStylesheet + { + public static partial XElement? Transform(XElement input); + } + + /// mml2omml.xsl: MathML to OMML. + internal sealed partial class Mml2ommlStylesheet + { + public static partial XElement? Transform(XElement input); + } + + /// ctop.xsl: content MathML to presentation MathML. + internal sealed partial class CtopStylesheet + { + public static partial XElement? Transform(XElement input); + } + + /// mtef2mml.xsl: the XML of MTEF records to MathML. + internal sealed partial class Mtef2mmlStylesheet + { + public static partial XElement? Transform(XElement input); + } +} diff --git a/src/SharpEquations/Internal/Xslt/XPathFunctions.cs b/src/SharpEquations/Internal/Xslt/XPathFunctions.cs new file mode 100644 index 0000000..e3576a0 --- /dev/null +++ b/src/SharpEquations/Internal/Xslt/XPathFunctions.cs @@ -0,0 +1,682 @@ +#nullable enable + +namespace SharpEquations.Internal.Xslt +{ + using System; + using System.Collections.Generic; + using System.Globalization; + using System.Linq; + using System.Text; + using System.Xml.Linq; + using SharpEquations.Internal.Text; + + internal enum Axis + { + Child, + Descendant, + DescendantOrSelf, + Parent, + Ancestor, + AncestorOrSelf, + FollowingSibling, + PrecedingSibling, + Self, + Attribute, + } + + internal enum Comparison + { + Equal, + NotEqual, + Less, + LessOrEqual, + Greater, + GreaterOrEqual, + } + + /// + /// The XPath 1.0 data model over LINQ to XML, and its functions, for the code generated from the stylesheets. + /// The values are strings, doubles, booleans, node-sets (lists in document order, without duplicates) and + /// result tree fragments; a value of unknown type is one of them as an object. Strings are counted, cut and + /// translated by code points, as XPath counts characters. + /// + internal static class XPathFunctions + { + public static readonly List Empty = new List(); + + private static readonly IComparer DocumentOrder = Comparer.Create(Compare); + + // conversions + + public static string Str(string value) => value; + + public static string Str(bool value) => value ? "true" : "false"; + + public static string Str(double value) + { + if (double.IsNaN(value)) + return "NaN"; + if (double.IsPositiveInfinity(value)) + return "Infinity"; + if (double.IsNegativeInfinity(value)) + return "-Infinity"; + if (value == 0) + return "0"; + var text = value.ToString("R", CultureInfo.InvariantCulture); + var exponent = text.IndexOf('E'); + return exponent < 0 ? text : ExpandExponent(text, exponent); + } + + private static string ExpandExponent(string text, int exponent) + { + var negative = text[0] == '-'; + var mantissa = text.Substring(negative ? 1 : 0, exponent - (negative ? 1 : 0)); + var power = int.Parse(text.Substring(exponent + 1), NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture); + var point = mantissa.IndexOf('.'); + var digits = point < 0 ? mantissa : mantissa.Remove(point, 1); + var integerDigits = (point < 0 ? mantissa.Length : point) + power; + string result; + if (integerDigits <= 0) + result = "0." + new string('0', -integerDigits) + digits; + else if (integerDigits >= digits.Length) + result = digits + new string('0', integerDigits - digits.Length); + else + result = digits.Substring(0, integerDigits) + "." + digits.Substring(integerDigits); + return negative ? "-" + result : result; + } + + public static string Str(List nodes) => nodes.Count == 0 ? "" : StringValue(nodes[0]); + + public static string Str(ResultFragment fragment) => fragment.Value; + + public static string Str(object value) + { + switch (value) + { + case string s: + return s; + case double d: + return Str(d); + case bool b: + return Str(b); + case List nodes: + return Str(nodes); + case ResultFragment fragment: + return fragment.Value; + default: + throw new InvalidOperationException("Not an XPath value: " + value); + } + } + + public static double Num(string value) + { + var s = value.Trim(' ', '\t', '\r', '\n'); + var i = 0; + if (i < s.Length && s[i] == '-') + i++; + var digits = 0; + while (i < s.Length && s[i] >= '0' && s[i] <= '9') + { + i++; + digits++; + } + if (i < s.Length && s[i] == '.') + { + i++; + while (i < s.Length && s[i] >= '0' && s[i] <= '9') + { + i++; + digits++; + } + } + if (digits == 0 || i != s.Length) + return double.NaN; + return double.Parse(s, NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture); + } + + public static double Num(double value) => value; + + public static double Num(bool value) => value ? 1 : 0; + + public static double Num(List nodes) => Num(Str(nodes)); + + public static double Num(ResultFragment fragment) => Num(fragment.Value); + + public static double Num(object value) + { + switch (value) + { + case double d: + return d; + case bool b: + return b ? 1 : 0; + default: + return Num(Str(value)); + } + } + + public static bool Bool(string value) => value.Length > 0; + + public static bool Bool(double value) => value != 0 && !double.IsNaN(value); + + public static bool Bool(bool value) => value; + + public static bool Bool(List nodes) => nodes.Count > 0; + + // a result tree fragment is a node-set of its root node + public static bool Bool(ResultFragment fragment) => true; + + public static bool Bool(object value) + { + switch (value) + { + case string s: + return s.Length > 0; + case double d: + return Bool(d); + case bool b: + return b; + case List nodes: + return nodes.Count > 0; + case ResultFragment _: + return true; + default: + throw new InvalidOperationException("Not an XPath value: " + value); + } + } + + public static List Nodes(object value) => + value as List ?? throw new InvalidOperationException("A node-set was expected, not " + (value is ResultFragment ? "a result tree fragment." : "a " + value.GetType().Name + ".")); + + /// The value of a predicate: a number selects the node at that position. + public static bool Predicate(object value, int position) => + value is double d ? d == position : Bool(value); + + public static string StringValue(XObject node) + { + switch (node) + { + case XElement element: + return element.Value; + case XDocument document: + return document.Root?.Value ?? ""; + case XAttribute attribute: + return attribute.Value; + case XText text: + return text.Value; + case XComment comment: + return comment.Value; + case XProcessingInstruction instruction: + return instruction.Data; + default: + return ""; + } + } + + // comparisons, as XPath 1.0 defines them for each pair of types + + public static bool Compare(object left, object right, Comparison comparison) + { + if (IsNodes(left)) + return IsNodes(right) ? CompareNodes(left, right, comparison) : CompareNodes(left, right, comparison, false); + if (IsNodes(right)) + return CompareNodes(right, left, comparison, true); + if (comparison == Comparison.Equal || comparison == Comparison.NotEqual) + { + bool equal; + if (left is bool || right is bool) + equal = Bool(left) == Bool(right); + else if (left is double || right is double) + equal = Num(left) == Num(right); + else + equal = Str(left) == Str(right); + return comparison == Comparison.Equal ? equal : !equal; + } + return Compare(Num(left), Num(right), comparison); + } + + public static bool Compare(double left, double right, Comparison comparison) + { + switch (comparison) + { + case Comparison.Equal: + return left == right; + case Comparison.NotEqual: + return left != right; + case Comparison.Less: + return left < right; + case Comparison.LessOrEqual: + return left <= right; + case Comparison.Greater: + return left > right; + default: + return left >= right; + } + } + + private static bool IsNodes(object value) => value is List || value is ResultFragment; + + private static IEnumerable StringValues(object value) => + value is ResultFragment fragment ? new[] { fragment.Value } : ((List)value).Select(StringValue); + + private static bool CompareNodes(object left, object right, Comparison comparison) + { + var rightValues = StringValues(right).ToList(); + foreach (var l in StringValues(left)) + { + foreach (var r in rightValues) + { + if (CompareStrings(l, r, comparison)) + return true; + } + } + return false; + } + + private static bool CompareNodes(object nodes, object other, Comparison comparison, bool swapped) + { + if (swapped) + comparison = Swap(comparison); + if (other is bool b) + { + var value = Bool(nodes); + return comparison == Comparison.Equal || comparison == Comparison.NotEqual + ? (value == b) == (comparison == Comparison.Equal) + : Compare(value ? 1 : 0, b ? 1 : 0, comparison); + } + if (other is double d) + return StringValues(nodes).Any(s => Compare(Num(s), d, comparison)); + var text = Str(other); + return StringValues(nodes).Any(s => CompareStrings(s, text, comparison)); + } + + private static bool CompareStrings(string left, string right, Comparison comparison) + { + switch (comparison) + { + case Comparison.Equal: + return left == right; + case Comparison.NotEqual: + return left != right; + default: + return Compare(Num(left), Num(right), comparison); + } + } + + private static Comparison Swap(Comparison comparison) + { + switch (comparison) + { + case Comparison.Less: + return Comparison.Greater; + case Comparison.LessOrEqual: + return Comparison.GreaterOrEqual; + case Comparison.Greater: + return Comparison.Less; + case Comparison.GreaterOrEqual: + return Comparison.LessOrEqual; + default: + return comparison; + } + } + + // node-sets + + public static List Single(XObject node) => new List(1) { node }; + + public static List Root(XObject node) + { + var root = node; + for (var parent = Parent(root); parent != null; parent = Parent(parent)) + root = parent; + return Single(root); + } + + public static XObject? Parent(XObject node) + { + switch (node) + { + case XAttribute attribute: + return attribute.Parent; + case XDocument _: + return null; + case XNode child: + return (XObject?)child.Parent ?? child.Document; + default: + return null; + } + } + + public static List Union(List left, List right) + { + if (left.Count == 0) + return right; + if (right.Count == 0) + return left; + var result = new List(left.Count + right.Count); + result.AddRange(left); + result.AddRange(right); + SortUnique(result); + return result; + } + + public static List Step(List input, Axis axis, NodeTest test) + { + var result = new List(); + var attributeAxis = axis == Axis.Attribute; + foreach (var context in input) + { + foreach (var node in Along(context, axis)) + { + if (test.Matches(node, attributeAxis)) + result.Add(node); + } + } + Order(result, input.Count, axis); + return result; + } + + public static List Step(List input, Axis axis, NodeTest test, params XPathPredicate[] predicates) + { + var result = new List(); + var attributeAxis = axis == Axis.Attribute; + var candidates = new List(); + foreach (var context in input) + { + candidates.Clear(); + foreach (var node in Along(context, axis)) + { + if (test.Matches(node, attributeAxis)) + candidates.Add(node); + } + result.AddRange(Apply(candidates, predicates)); + } + Order(result, input.Count, axis); + return result; + } + + public static List Filter(List nodes, XPathPredicate predicate) + { + var result = new List(); + for (var i = 0; i < nodes.Count; i++) + { + if (predicate(nodes[i], i + 1, nodes.Count)) + result.Add(nodes[i]); + } + return result; + } + + private static List Apply(List candidates, XPathPredicate[] predicates) + { + var nodes = candidates; + foreach (var predicate in predicates) + nodes = Filter(nodes, predicate); + return ReferenceEquals(nodes, candidates) ? new List(candidates) : nodes; + } + + private static bool IsReverse(Axis axis) => + axis == Axis.Ancestor || axis == Axis.AncestorOrSelf || axis == Axis.PrecedingSibling; + + private static void Order(List nodes, int contexts, Axis axis) + { + if (contexts > 1) + SortUnique(nodes); + else if (IsReverse(axis)) + nodes.Reverse(); + } + + private static IEnumerable Along(XObject node, Axis axis) + { + switch (axis) + { + case Axis.Child: + return Children(node); + case Axis.Descendant: + return Descendants(node); + case Axis.DescendantOrSelf: + return new[] { node }.Concat(Descendants(node)); + case Axis.Parent: + var parent = Parent(node); + return parent == null ? Enumerable.Empty() : new[] { parent }; + case Axis.Ancestor: + return Ancestors(node); + case Axis.AncestorOrSelf: + return new[] { node }.Concat(Ancestors(node)); + case Axis.FollowingSibling: + return node is XNode following && !(node is XDocument) ? following.NodesAfterSelf().Where(IsInModel) : Enumerable.Empty(); + case Axis.PrecedingSibling: + return node is XNode preceding && !(node is XDocument) ? preceding.NodesBeforeSelf().Reverse().Where(IsInModel) : Enumerable.Empty(); + case Axis.Self: + return new[] { node }; + default: + return node is XElement element ? element.Attributes().Where(a => !a.IsNamespaceDeclaration) : Enumerable.Empty(); + } + } + + public static List ChildNodes(XObject node) => new List(Children(node)); + + /// + /// Whether a node matches a step of a pattern whose predicates count positions: whether the step, from the + /// parent of the node, selects it. + /// + public static bool MatchesStep(XObject node, Axis axis, NodeTest test, params XPathPredicate[] predicates) + { + var parent = Parent(node); + return parent != null && test.Matches(node, axis == Axis.Attribute) && Step(Single(parent), axis, test, predicates).Contains(node); + } + + public static IEnumerable Children(XObject node) => + node is XContainer container ? container.Nodes().Where(IsInModel) : Enumerable.Empty(); + + private static IEnumerable Descendants(XObject node) => + node is XContainer container ? container.DescendantNodes().Where(IsInModel) : Enumerable.Empty(); + + private static IEnumerable Ancestors(XObject node) + { + for (var parent = Parent(node); parent != null; parent = Parent(parent)) + yield return parent; + } + + // the document type and the white space around the document element are not nodes of XPath + private static bool IsInModel(XNode node) => + !(node is XDocumentType) && !(node is XText && node.Parent == null); + + private static void SortUnique(List nodes) + { + nodes.Sort(DocumentOrder); + var write = 0; + for (var read = 0; read < nodes.Count; read++) + { + if (write == 0 || !ReferenceEquals(nodes[read], nodes[write - 1])) + nodes[write++] = nodes[read]; + } + nodes.RemoveRange(write, nodes.Count - write); + } + + private static int Compare(XObject? left, XObject? right) + { + if (ReferenceEquals(left, right)) + return 0; + // an attribute comes after its element and before the children of the element, so that two nodes are + // in the order of their elements, unless they have the same + var leftOwner = left is XAttribute la ? la.Parent! : (XNode)left!; + var rightOwner = right is XAttribute ra ? ra.Parent! : (XNode)right!; + if (!ReferenceEquals(leftOwner, rightOwner)) + return XNode.CompareDocumentOrder(leftOwner, rightOwner); + switch (left) + { + case XAttribute a when right is XAttribute b: + return AttributeIndex(a).CompareTo(AttributeIndex(b)); + case XAttribute _: + return 1; + default: + return -1; + } + } + + private static int AttributeIndex(XAttribute attribute) + { + var index = 0; + for (var a = attribute.PreviousAttribute; a != null; a = a.PreviousAttribute) + index++; + return index; + } + + // the functions of the XPath core library + + public static double Count(List nodes) => nodes.Count; + + public static double Sum(List nodes) => nodes.Sum(n => Num(StringValue(n))); + + public static string LocalName(List nodes) => nodes.Count == 0 ? "" : LocalName(nodes[0]); + + public static string LocalName(XObject node) + { + switch (node) + { + case XElement element: + return element.Name.LocalName; + case XAttribute attribute: + return attribute.Name.LocalName; + case XProcessingInstruction instruction: + return instruction.Target; + default: + return ""; + } + } + + public static string NamespaceUri(List nodes) => nodes.Count == 0 ? "" : NamespaceUri(nodes[0]); + + public static string NamespaceUri(XObject node) + { + switch (node) + { + case XElement element: + return element.Name.NamespaceName; + case XAttribute attribute: + return attribute.Name.NamespaceName; + default: + return ""; + } + } + + public static string Name(List nodes) => nodes.Count == 0 ? "" : Name(nodes[0]); + + public static string Name(XObject node) + { + XName name; + XElement? scope; + switch (node) + { + case XElement element: + name = element.Name; + scope = element; + break; + case XAttribute attribute: + name = attribute.Name; + scope = attribute.Parent; + break; + default: + return LocalName(node); + } + var prefix = name.Namespace == XNamespace.None ? null : scope?.GetPrefixOfNamespace(name.Namespace); + return string.IsNullOrEmpty(prefix) ? name.LocalName : prefix + ":" + name.LocalName; + } + + public static bool StartsWith(string s, string prefix) => s.StartsWith(prefix, StringComparison.Ordinal); + + public static bool Contains(string s, string part) => s.Contains(part); + + public static string SubstringBefore(string s, string part) + { + var index = s.IndexOf(part, StringComparison.Ordinal); + return index < 0 ? "" : s.Substring(0, index); + } + + public static string SubstringAfter(string s, string part) + { + var index = s.IndexOf(part, StringComparison.Ordinal); + return index < 0 ? "" : s.Substring(index + part.Length); + } + + public static string Substring(string s, double start) => Substring(s, start, double.PositiveInfinity); + + public static string Substring(string s, double start, double length) + { + var first = Round(start); + var end = first + Round(length); + var result = new StringBuilder(); + var position = 0; + foreach (var rune in CodePoints.Of(s)) + { + position++; + if (position >= first && position < end) + result.Append(rune.ToString()); + } + return result.ToString(); + } + + public static double StringLength(string s) => CodePoints.Of(s).Count(); + + public static string NormalizeSpace(string s) + { + var result = new StringBuilder(s.Length); + var space = false; + foreach (var c in s) + { + if (c == ' ' || c == '\t' || c == '\r' || c == '\n') + { + space = result.Length > 0; + continue; + } + if (space) + result.Append(' '); + space = false; + result.Append(c); + } + return result.ToString(); + } + + public static string Translate(string s, string from, string to) + { + var source = CodePoints.Of(from).ToList(); + var target = CodePoints.Of(to).ToList(); + var result = new StringBuilder(s.Length); + foreach (var rune in CodePoints.Of(s)) + { + var index = source.IndexOf(rune); + if (index < 0) + result.Append(rune.ToString()); + else if (index < target.Count) + result.Append(target[index].ToString()); + } + return result.ToString(); + } + + public static double Round(double value) + { + if (double.IsNaN(value) || double.IsInfinity(value) || value == 0) + return value; + if (value < 0 && value >= -0.5) + return -0.0; + return Math.Floor(value + 0.5); + } + + // XSLT + + /// The expanded name of a QName computed at run time, with the namespaces in scope where it was written. + public static XName ResolveName(string qname, string defaultNamespace, string[] prefixesAndNamespaces) + { + var colon = qname.IndexOf(':'); + if (colon < 0) + return XName.Get(qname, defaultNamespace); + var prefix = qname.Substring(0, colon); + for (var i = 0; i < prefixesAndNamespaces.Length; i += 2) + { + if (prefixesAndNamespaces[i] == prefix) + return XName.Get(qname.Substring(colon + 1), prefixesAndNamespaces[i + 1]); + } + throw new InvalidOperationException("The prefix '" + prefix + "' is not declared."); + } + } +} diff --git a/src/SharpEquations/Internal/Xslt/XsltOutput.cs b/src/SharpEquations/Internal/Xslt/XsltOutput.cs new file mode 100644 index 0000000..0d820e9 --- /dev/null +++ b/src/SharpEquations/Internal/Xslt/XsltOutput.cs @@ -0,0 +1,252 @@ +#nullable enable + +namespace SharpEquations.Internal.Xslt +{ + using System; + using System.Collections.Generic; + using System.IO; + using System.Linq; + using System.Text; + using System.Xml; + using System.Xml.Linq; + + /// + /// The result tree a generated stylesheet builds, as LINQ to XML, with the result tree fragments and the + /// attribute values its instructions build on the way. It gives what XslCompiledTransform gives when its output + /// is written as text and parsed again: the text written with disable-output-escaping is markup, which takes + /// effect with the text next to it (a character reference may come in pieces), and empty text leaves an empty + /// element with an end tag. + /// + internal sealed class XsltOutput + { + private static readonly XName Container = XName.Get("fragment"); + + private readonly Stack open = new Stack(); + private readonly Stack roots = new Stack(); + private readonly XElement result = new XElement(Container); + + // the text written since the last node, as it is and as markup (escaped, with the unescaped text as it is) + private readonly StringBuilder text = new StringBuilder(); + private readonly StringBuilder markup = new StringBuilder(); + private bool hasText; + private bool hasMarkup; + + public XsltOutput() + { + open.Push(result); + roots.Push(result); + } + + private XContainer Top => open.Peek(); + + public void StartElement(XName name) + { + Flush(); + var element = new XElement(name); + Top.Add(element); + open.Push(element); + } + + public void EndElement() + { + Flush(); + open.Pop(); + } + + public void Attribute(XName name, string value) + { + Flush(); + if (!(Top is XElement element) || ReferenceEquals(element, roots.Peek())) + throw new InvalidOperationException("An attribute can be added only to an element."); + if (element.Nodes().Any(n => !(n is XText t) || t.Value.Length > 0)) + throw new InvalidOperationException("An attribute cannot be added to an element after its children."); + element.SetAttributeValue(name, value); + } + + public void Text(string value) + { + hasText = true; + text.Append(value); + foreach (var c in value) + { + switch (c) + { + case '&': + markup.Append("&"); + break; + case '<': + markup.Append("<"); + break; + case '>': + markup.Append(">"); + break; + default: + markup.Append(c); + break; + } + } + } + + /// Text written with disable-output-escaping. + public void RawText(string value) + { + hasText = true; + hasMarkup = true; + text.Append(value); + markup.Append(value); + } + + private void Flush() + { + if (!hasText) + return; + var top = Top; + if (hasMarkup) + { + var settings = new XmlReaderSettings { DtdProcessing = DtdProcessing.Prohibit, XmlResolver = null }; + using (var reader = XmlReader.Create(new StringReader("" + markup + ""), settings)) + top.Add(XElement.Load(reader, LoadOptions.PreserveWhitespace).Nodes().ToList()); + } + else if (text.Length > 0 || (top is XElement element && element.IsEmpty && !ReferenceEquals(element, roots.Peek()))) + { + top.Add(text.ToString()); + } + text.Clear(); + markup.Clear(); + hasText = false; + hasMarkup = false; + } + + public void Comment(string value) + { + Flush(); + Top.Add(new XComment(value)); + } + + public void ProcessingInstruction(string target, string data) + { + Flush(); + Top.Add(new XProcessingInstruction(target, data)); + } + + public void CopyOf(List nodes) + { + foreach (var node in nodes) + Copy(node); + } + + public void CopyOf(ResultFragment fragment) + { + foreach (var node in fragment.Root.Nodes()) + Copy(node); + } + + public void CopyOf(string value) => Text(value); + + public void CopyOf(object value) + { + switch (value) + { + case List nodes: + CopyOf(nodes); + break; + case ResultFragment fragment: + CopyOf(fragment); + break; + default: + Text(XPathFunctions.Str(value)); + break; + } + } + + private void Copy(XObject node) + { + switch (node) + { + case XDocument document: + foreach (var child in XPathFunctions.Children(document)) + Copy(child); + break; + case XElement element: + Flush(); + Top.Add(new XElement(element)); + break; + case XAttribute attribute: + Attribute(attribute.Name, attribute.Value); + break; + case XText t: + Text(t.Value); + break; + case XComment comment: + Comment(comment.Value); + break; + case XProcessingInstruction instruction: + ProcessingInstruction(instruction.Target, instruction.Data); + break; + } + } + + /// The shallow copy of xsl:copy: the node without its children, open for content if it is an element. + public bool StartCopy(XObject node) + { + switch (node) + { + case XElement element: + StartElement(element.Name); + return true; + case XDocument _: + return false; + default: + Copy(node); + return false; + } + } + + public void BeginCapture() + { + Flush(); + var root = new XElement(Container); + open.Push(root); + roots.Push(root); + } + + public ResultFragment EndFragment() + { + Flush(); + open.Pop(); + return new ResultFragment(roots.Pop()); + } + + /// The text captured, for an attribute or a comment, where disable-output-escaping does not apply. + public string EndString() + { + hasMarkup = false; + Flush(); + open.Pop(); + return roots.Pop().Value; + } + + /// The document element of the result, or null if the result is only white space. + /// The result is not a document. + public XElement? Result() + { + Flush(); + XElement? element = null; + foreach (var node in result.Nodes()) + { + switch (node) + { + case XElement e when element == null: + element = e; + break; + case XElement _: + throw new XmlException("The result of the stylesheet has more than one document element."); + case XText t when t.Value.Trim().Length > 0: + throw new XmlException("The result of the stylesheet has text outside the document element."); + } + } + element?.Remove(); + return element; + } + } +} diff --git a/src/SharpEquations/SharpEquations.csproj b/src/SharpEquations/SharpEquations.csproj index 7186b17..7c43b0b 100644 --- a/src/SharpEquations/SharpEquations.csproj +++ b/src/SharpEquations/SharpEquations.csproj @@ -11,10 +11,12 @@ relative + + true - @@ -38,6 +40,12 @@ + + + + + + diff --git a/tests/SharpEquations.Tests/Omml/OmmlAstralTests.cs b/tests/SharpEquations.Tests/Omml/OmmlAstralTests.cs index 41ddbb6..1c1d382 100644 --- a/tests/SharpEquations.Tests/Omml/OmmlAstralTests.cs +++ b/tests/SharpEquations.Tests/Omml/OmmlAstralTests.cs @@ -14,11 +14,11 @@ namespace SharpEquations.Tests.Omml /// /// The mathematical alphanumerics are written as Word writes them, the plain letter with its style (𝒢 is a G with /// m:scr), before mml2omml.xsl sees them, so how the stylesheet counts characters does not matter for them. An - /// emoji reaches the stylesheet as it is, and the stylesheet, run by XslCompiledTransform, counts UTF-16 units - /// where XPath counts code points: 🍕, one character, counts as two, so its mi is taken for a name of several - /// letters and is written upright (m:sty p), in a run of its own. The tests marked as depending on UTF-16 counting - /// pin that behaviour: an implementation that counts code points writes 🍕 as an italic identifier, without m:rPr, - /// in one run with the characters around it, and reads it back to the same equation. + /// emoji reaches the stylesheet as it is, and the stylesheet, compiled to C# by SharpEquations.XsltGenerator, + /// counts code points as XPath does (XslCompiledTransform counted UTF-16 units, so that 🍕 counted as two + /// characters and was written upright, in a run of its own): 🍕 is written as an italic identifier, without m:rPr, + /// in one run with the characters around it, and reads back to the same equation. The tests marked as counting + /// code points pin that behaviour. /// public class OmmlAstralTests { @@ -78,32 +78,30 @@ public async Task WritesTheDerivativesOfTheCorpus() } /// - /// Depends on UTF-16 counting: 🍕 counts as two characters in mml2omml.xsl and is written upright, in a run of - /// its own. Counting code points, it would be <m:r><m:t>🍕</m:t></m:r>, and - /// -:x+🍕 in one run. + /// Counting code points: 🍕 is one character in mml2omml.xsl, an italic identifier written without m:rPr, in + /// one run with the characters around it. /// [Test] - [Arguments("🍕", "🍕")] - [Arguments("𝑥+🍕", "x+🍕")] - public async Task WritesEmojiUprightCountingUtf16Units(string unicodeMath, string omml) + [Arguments("🍕", "🍕")] + [Arguments("𝑥+🍕", "x+🍕")] + public async Task WritesEmojiItalicCountingCodePoints(string unicodeMath, string omml) { await Assert.That(Compact(Equation.Parse(unicodeMath, EquationFormat.UnicodeMath).ToOmml())).IsEqualTo(omml); } /// - /// Depends on UTF-16 counting: each 🍕 of the corpus example is written upright (m:sty p). Counting code points, - /// the runs would be nor=:pizza -:🍕 -:🍕. + /// Counting code points: each 🍕 of the corpus example is written as an italic identifier, without m:rPr. /// [Test] - public async Task WritesThePizzaOfTheCorpusCountingUtf16Units() + public async Task WritesThePizzaOfTheCorpusCountingCodePoints() { var omml = Equation.Parse("(\"pizza\"^🍕)^🍕", EquationFormat.UnicodeMath).ToOmml(); - await Assert.That(Runs(omml)).IsEqualTo("nor=:pizza sty=p:🍕 sty=p:🍕"); + await Assert.That(Runs(omml)).IsEqualTo("nor=:pizza -:🍕 -:🍕"); await Assert.That(Compact(omml)).IsEqualTo("" + "pizza" - + "🍕" + + "🍕" + "" - + "🍕"); + + "🍕"); } [Test] @@ -136,22 +134,20 @@ public async Task RoundTripsAlphanumericsThroughOmml(string unicodeMath) } /// - /// Depends on UTF-16 counting: the upright 🍕 that mml2omml.xsl writes reads back with - /// mathvariant="normal", so the first trip through OMML gives another equation, which then stays the same. - /// The UnicodeMath of both is the same. Counting code points, the first trip would give the same equation. + /// Counting code points: the italic 🍕 that mml2omml.xsl writes reads back as it was, so the first trip through + /// OMML gives the same equation. /// [Test] - [Arguments("🍕", "🍕")] - [Arguments("𝑥+🍕", "x+🍕")] - [Arguments("(\"pizza\"^🍕)^🍕", "(pizza🍕)🍕")] - public async Task RoundTripsEmojiThroughOmmlCountingUtf16Units(string unicodeMath, string mathML) + [Arguments("🍕", "🍕")] + [Arguments("𝑥+🍕", "x+🍕")] + [Arguments("(\"pizza\"^🍕)^🍕", "(pizza🍕)🍕")] + public async Task RoundTripsEmojiThroughOmmlCountingCodePoints(string unicodeMath, string mathML) { var equation = Equation.Parse(unicodeMath, EquationFormat.UnicodeMath); var back = Equation.Parse(equation.ToOmml(), EquationFormat.Omml); await Assert.That(back.ToMathML().ToString(SaveOptions.DisableFormatting)).IsEqualTo(MathML(mathML)); - await Assert.That(back).IsNotEqualTo(equation); + await Assert.That(back).IsEqualTo(equation); await Assert.That(back.ToString(EquationFormat.UnicodeMath)).IsEqualTo(equation.ToString(EquationFormat.UnicodeMath)); - await Assert.That(Equation.Parse(back.ToOmml(), EquationFormat.Omml)).IsEqualTo(back); } [Test] @@ -222,17 +218,16 @@ await Assert.That(back.ToMathML().Descendants(MathMLNamespaces.MathML + "msub"). } /// - /// Depends on UTF-16 counting: an italic 🍕 of OMML (a run without m:sty) is written back upright, so it reads - /// back as another equation. Counting code points, it would be written as it was read. + /// Counting code points: an italic 🍕 of OMML (a run without m:sty) is written back as it was read. /// [Test] - public async Task ItalicEmojiOfOmmlComesBackUprightCountingUtf16Units() + public async Task ItalicEmojiOfOmmlComesBackItalicCountingCodePoints() { var equation = Equation.Parse("🍕", EquationFormat.Omml); - await Assert.That(Runs(equation.ToOmml())).IsEqualTo("sty=p:🍕"); + await Assert.That(Runs(equation.ToOmml())).IsEqualTo("-:🍕"); var back = Equation.Parse(equation.ToOmml(), EquationFormat.Omml); - await Assert.That(back).IsNotEqualTo(equation); - await Assert.That(back.ToMathML().ToString(SaveOptions.DisableFormatting)).IsEqualTo(MathML("🍕")); + await Assert.That(back).IsEqualTo(equation); + await Assert.That(back.ToMathML().ToString(SaveOptions.DisableFormatting)).IsEqualTo(MathML("🍕")); } } } diff --git a/tests/SharpEquations.Tests/Xslt/CompiledStylesheetTests.cs b/tests/SharpEquations.Tests/Xslt/CompiledStylesheetTests.cs new file mode 100644 index 0000000..4d6d4be --- /dev/null +++ b/tests/SharpEquations.Tests/Xslt/CompiledStylesheetTests.cs @@ -0,0 +1,159 @@ +#nullable enable + +namespace SharpEquations.Tests.Xslt +{ + using System; + using System.Collections.Concurrent; + using System.Collections.Generic; + using System.IO; + using System.Linq; + using System.Threading.Tasks; + using System.Xml; + using System.Xml.Linq; + using System.Xml.Xsl; + using SharpEquations.Internal; + using SharpEquations.Internal.Mtef; + using SharpEquations.Internal.Xslt; + using SharpEquations.Tests.Property; + + /// + /// The stylesheets compiled to C# by SharpEquations.XsltGenerator give what XslCompiledTransform gives, for the + /// equations of the corpora: the OMML of Word documents and of the UnicodeMathML examples, their MathML, and the + /// MTEF of the test data. + /// + public class CompiledStylesheetTests + { + private static readonly XNamespace M = "http://schemas.openxmlformats.org/officeDocument/2006/math"; + + private static readonly ConcurrentDictionary Stylesheets = new ConcurrentDictionary(); + + /// The stylesheet of the sources run by XslCompiledTransform, as SharpEquations ran it before it was compiled to C#. + private static XElement? Reference(string name, XElement input) + { + var stylesheet = Stylesheets.GetOrAdd(name, n => + { + using var stream = File.OpenRead(Path.Combine(TestData.Path(), "..", "..", "src", "SharpEquations", "Resources", n)); + using var reader = XmlReader.Create(stream, new XmlReaderSettings { DtdProcessing = DtdProcessing.Prohibit, XmlResolver = null }); + var transform = new XslCompiledTransform(); + transform.Load(reader, XsltSettings.Default, null); + return transform; + }); + var output = new StringWriter(); + using (var reader = new XDocument(input).CreateReader()) + stylesheet.Transform(reader, null, output); + var text = output.ToString(); + if (text.Trim().Length == 0) + return null; + using var result = XmlReader.Create(new StringReader(text), new XmlReaderSettings { DtdProcessing = DtdProcessing.Prohibit, XmlResolver = null }); + return XElement.Load(result, LoadOptions.PreserveWhitespace); + } + + // the element as text, without the namespace declarations, which the two write differently + private static string Canonical(XElement? element) + { + if (element == null) + return "(null)"; + var copy = new XElement(element); + foreach (var e in copy.DescendantsAndSelf()) + e.Attributes().Where(a => a.IsNamespaceDeclaration).Remove(); + return copy.ToString(SaveOptions.DisableFormatting); + } + + private static string Run(Func transform) + { + try + { + return Canonical(transform()); + } + catch (Exception e) + { + return e.GetType().Name; + } + } + + // the differences for the inputs without characters outside the basic plane, and the number of those with + private static (List Differences, int Astral) Differences(string name, Func compiled, IEnumerable<(string Label, XElement Input)> inputs) + { + var differences = new List(); + var astral = 0; + foreach (var (label, input) in inputs) + { + var expected = Run(() => Reference(name, new XElement(input))!); + var actual = Run(() => compiled(new XElement(input))!); + if (expected == actual) + continue; + if (input.DescendantNodesAndSelf().OfType().Any(t => t.Value.Any(char.IsSurrogate))) + astral++; + else + differences.Add(label + "\n expected " + Around(expected, actual) + "\n actual " + Around(actual, expected)); + } + return (differences, astral); + } + + // the text around the first character where it differs from the other + private static string Around(string text, string other) + { + var i = 0; + while (i < text.Length && i < other.Length && text[i] == other[i]) + i++; + var start = Math.Max(0, i - 120); + return (start > 0 ? "…" : "") + text.Substring(start, Math.Min(text.Length - start, 240)); + } + + private static IEnumerable<(string Label, XElement Omml)> WordOmml() => + Directory.GetFiles(TestData.Path("omml"), "*.omml") + .OrderBy(f => f, StringComparer.Ordinal) + .SelectMany(f => XElement.Load(f).DescendantsAndSelf(M + "oMath").Select((e, i) => (Path.GetFileName(f) + "#" + i, e))); + + private static IEnumerable<(string Label, Equation Equation)> Equations() => + UnicodeMathCorpusTests.Examples() + .Select(e => (e.input, Equation: Equation.TryParse(e.input, EquationFormat.UnicodeMath, out var equation) ? equation : null)) + .Where(e => e.Equation != null) + .Select(e => (e.input, e.Equation!)) + .Concat(WordOmml().Select(o => (o.Label, Equation.Parse(o.Omml, EquationFormat.Omml)))); + + private static async Task AssertNoDifferences((List Differences, int Astral) result, int inputs) + { + await Assert.That(inputs).IsGreaterThan(100); + Console.WriteLine(result.Astral + " of " + inputs + " differ, with characters outside the basic plane"); + await Assert.That(result.Differences.Count).IsEqualTo(0).Because(string.Join("\n", result.Differences.Take(10))); + } + + [Test] + public async Task Omml2MathML() + { + var inputs = WordOmml().Concat(Equations().Select(e => (e.Label, e.Equation.ToOmml()))).ToList(); + await AssertNoDifferences(Differences("omml2mml.xsl", Omml2mmlStylesheet.Transform, inputs), inputs.Count); + } + + [Test] + public async Task MathML2Omml() + { + var inputs = Equations().Select(e => (e.Label, e.Equation.ToMathML())).ToList(); + await AssertNoDifferences(Differences("mml2omml.xsl", Mml2ommlStylesheet.Transform, inputs), inputs.Count); + } + + [Test] + public async Task Mtef2MathML() + { + var inputs = new List<(string, XElement)>(); + foreach (var file in Directory.GetFiles(TestData.Path("mtef"), "*.*", SearchOption.AllDirectories).OrderBy(f => f, StringComparer.Ordinal)) + { + var data = File.ReadAllBytes(file).AsSpan(); + if (MtefParser.HasEquationNativeHeader(data)) + data = data.Slice(data[0] | data[1] << 8); + try + { + var xml = MtefReader.Read(data); + MtefPreprocessor.Prepare(xml); + inputs.Add((Path.GetFileName(file), xml)); + } + catch (Exception) + { + // not MTEF + } + } + await AssertNoDifferences(Differences("mtef2mml.xsl", Mtef2mmlStylesheet.Transform, inputs), inputs.Count); + } + } +}