Quantcast

rule reflection

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

rule reflection

jms_nh
Is there any way to examine the structure of a BaseParser once it has been created? I would like to write some Java code to analyze the grammar (specifically print it in a quasi-BNF form, formatted the way I like). The only way I can do this now is to parse the source code itself and hope that I've got it right.

--Jason
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rule reflection

Ken Wenzel
Hallo Jason,

Parboiled applies the visitor pattern to traverse the resulting matcher tree.

You can either implement the interface org.parboiled.matchervisitors.MatcherVisitor
or extend org.parboiled.matchervisitors.DefaultMatcherVisitor to implement
your transformation logic.

Then you can simply do:

MyParser parser = Parboiled.createParser(MyParser.class);
((Matcher)parser.EntryRule()).accept(new MyMatcherVisitor());

I hope this helps.

Best regards,

Ken
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rule reflection

jms_nh
thanks, that was a good starting point.

What are EmptyMatcher and NothingMatcher, and when would they ever be used?

just for posterity's sake, I've got a preliminary grammar-printing class (see below, it has a few unfinished loose ends). Working directly on the Matcher rules themselves was awkward, so I wrote a RuleConverter class to transform a graph of Matcher objects to a list of (simpler) RuleExpressions, which can be printed directly via toString(), or visited for custom presentation.

=====

package com.example.parser;

import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.LinkedList;
import java.util.List;
import java.util.Queue;
import java.util.Set;
import org.parboiled.matchers.ActionMatcher;
import org.parboiled.matchers.AnyMatcher;
import org.parboiled.matchers.AnyOfMatcher;
import org.parboiled.matchers.CharIgnoreCaseMatcher;
import org.parboiled.matchers.CharMatcher;
import org.parboiled.matchers.CharRangeMatcher;
import org.parboiled.matchers.CustomMatcher;
import org.parboiled.matchers.EmptyMatcher;
import org.parboiled.matchers.FirstOfMatcher;
import org.parboiled.matchers.Matcher;
import org.parboiled.matchers.NothingMatcher;
import org.parboiled.matchers.OneOrMoreMatcher;
import org.parboiled.matchers.OptionalMatcher;
import org.parboiled.matchers.SequenceMatcher;
import org.parboiled.matchers.StringMatcher;
import org.parboiled.matchers.TestMatcher;
import org.parboiled.matchers.TestNotMatcher;
import org.parboiled.matchers.ZeroOrMoreMatcher;
import org.parboiled.matchervisitors.MatcherVisitor;

public class RuleConverter {
        final static public ExpressionVisitor<String> stringifier
                = new ExpressionVisitor<String>()
        {
                public String visit(UnaryExpression expr) {
                        return expr.getModifier().getPrefix()+getSubstring(expr.getSubject())+expr.getModifier().getSuffix();
                }

                public String visit(FirstOfExpression expr) {
                        return join(expr.getChildren(), "", " | ", "");
                }

                public String visit(SequenceExpression expr) {
                        return join(expr.getChildren(), "{", ", ", "}");
                }

                public String visit(CharExpression expr) {
                        return expr.getChars();
                }

                public String visit(PlaceholderExpression expr) {
                        return expr.getName();
                }

                public String visit(NamedExpression expr) {
                        return expr.getName()+" ::= "+expr.getSubject().accept(this);
                }

                /* ------------------------------------------------- */
               
                private String getSubstring(RuleExpression subject) {
                        String baseString = subject.accept(this);
                        if (subject instanceof SequenceExpression)
                                return baseString;
                        return "("+baseString+")";
                }

                final protected String join(List<RuleExpression> list, String prefix, String separator, String suffix)
                {
                        StringBuilder sb = new StringBuilder();
                        sb.append(prefix);
                        int L = list.size();
                        int i = 0;
                        for (RuleExpression re : list)
                        {
                                sb.append(re.accept(this));
                                if (++i < L)
                                {
                                        sb.append(separator);
                                }
                        }
                        sb.append(suffix);
                        return sb.toString();
                }
               
        };
       
        public interface RuleExpression
        {
                public <T> T accept(ExpressionVisitor<T> visitor);
        }
        static abstract public class BaseRuleExpression implements RuleExpression
        {
                @Override public String toString() { return accept(stringifier); }
        }
        public interface ExpressionVisitor<T>
        {
                public T visit(UnaryExpression expr);
                public T visit(FirstOfExpression expr);
                public T visit(SequenceExpression expr);
                public T visit(CharExpression expr);
                public T visit(PlaceholderExpression expr);
                public T visit(NamedExpression expr);
        }
        enum UnaryModifier {
                ZERO_OR_MORE("*","","*"),
                ONE_OR_MORE("+","","+"),
                OPTIONAL("?","","?"),
                TEST("&","&",""),
                TEST_NOT("!","!",""),
                ;
               
                final private String sym, pre, post;
                private UnaryModifier(String sym, String pre, String post) { this.sym = sym; this.pre = pre; this.post = post; }
                public String getPrefix() { return this.pre; }
                public String getSuffix() { return this.post; }
                public String getSymbol() { return this.sym; }

                public UnaryExpression wrap(RuleExpression subject)
                {
                        return new UnaryExpression(this, subject);
                }
        }
        static public class UnaryExpression extends BaseRuleExpression
        {
                final private UnaryModifier modifier;
                final private RuleExpression subject;
               
                public UnaryModifier getModifier() { return this.modifier; }
                public RuleExpression getSubject() { return this.subject; }
                private UnaryExpression(UnaryModifier modifier, RuleExpression subject) {
                        this.modifier = modifier; this.subject = subject;
                }
               
                static public UnaryExpression zeroOrMore(RuleExpression subject) { return UnaryModifier.ZERO_OR_MORE.wrap(subject); }
                static public UnaryExpression optional(RuleExpression subject) { return UnaryModifier.OPTIONAL.wrap(subject); }
                static public UnaryExpression oneOrMore(RuleExpression subject) { return UnaryModifier.ONE_OR_MORE.wrap(subject); }
                static public UnaryExpression test(RuleExpression subject) { return UnaryModifier.TEST.wrap(subject); }
                static public UnaryExpression testNot(RuleExpression subject) { return UnaryModifier.TEST_NOT.wrap(subject); }
                public String asSubstring() { return "("+toString()+")"; }
               
                public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
        }
       
        abstract static public class ListExpression extends BaseRuleExpression
        {
                final private List<RuleExpression> list;
                public ListExpression(List<RuleExpression> list)
                {
                        this.list = Collections.unmodifiableList(new ArrayList<RuleExpression>(list));
                }
                public List<RuleExpression> getChildren() { return this.list; }
        }
        static public class SequenceExpression extends ListExpression
        {
                private SequenceExpression(List<RuleExpression> list) { super(list); }
                static public RuleExpression of(List<RuleExpression> list)
                {
                        if (list.size() == 1)
                                return list.get(0);
                        else
                                return new SequenceExpression(list);
                }
                public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
        }
        static public class FirstOfExpression extends ListExpression
        {
                private FirstOfExpression(List<RuleExpression> list) { super(list); }
                static public RuleExpression of(List<RuleExpression> list)
                {
                        if (list.size() == 1)
                                return list.get(0);
                        else
                                return new FirstOfExpression(list);
                }
                public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
        }
        static public class CharExpression extends BaseRuleExpression
        {
                final private String chars;
                private CharExpression(String chars) { this.chars = chars; }
                public String getChars() { return this.chars; }
                static public CharExpression of(Matcher m)
                {
                        return new CharExpression(m.getLabel());
                }
                static public CharExpression of(CharRangeMatcher m)
                {
                        return new CharExpression("'"+m.cLow+"'..'"+m.cHigh+"'");
                }
                public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
        }
        static public class PlaceholderExpression extends BaseRuleExpression
        {
                final private String name;
                public PlaceholderExpression(String name) {
                        this.name = name;
                }
                public String getName() { return this.name; }
                public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
        }
        static public class NamedExpression extends PlaceholderExpression
        {
                final private RuleExpression subject;
                public NamedExpression(String name, RuleExpression subject) {
                        super(name);
                        this.subject = subject;
                }
                public RuleExpression getSubject() { return this.subject; }
                @Override public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
        }
       
        abstract static class Helper implements MatcherVisitor<RuleExpression> {
                       
                abstract public void defer(Matcher m);
               
                public RuleExpression visit(ActionMatcher arg0) { return null; }
                public RuleExpression visit(AnyMatcher arg0) {
                        System.out.println("any{");
                        System.out.println("}");
                        return null;
                }

                public RuleExpression visit(CharIgnoreCaseMatcher arg0) {
                        return CharExpression.of(arg0);
                }

                public RuleExpression visit(CharMatcher arg0) {
                        return CharExpression.of(arg0);
                }

                public RuleExpression visit(CustomMatcher arg0) {
                        System.out.println("custom:"+arg0);
                        return null;
                }

                public RuleExpression visit(CharRangeMatcher arg0) {
                        return CharExpression.of(arg0);
                }

                public RuleExpression visit(AnyOfMatcher arg0) {
                        return CharExpression.of(arg0);
                }

                public RuleExpression visit(EmptyMatcher arg0) {
                        System.out.println("empty:"+describe(arg0));
                        return null;
                }

                public RuleExpression visit(FirstOfMatcher arg0) {
                        return annotateLabel(arg0, FirstOfExpression.of(visitChildren(arg0)));
                }


                public RuleExpression visit(NothingMatcher arg0) {
                        System.out.println("nothing{}");
                        return null;
                }

                public RuleExpression visit(OneOrMoreMatcher arg0) {
                        return annotateLabel(arg0, UnaryExpression.oneOrMore(visitChild(arg0)));
                }

                public RuleExpression visit(OptionalMatcher arg0) {
                        return annotateLabel(arg0, UnaryExpression.optional(visitChild(arg0)));
                }

                public RuleExpression annotateLabel(Matcher m, RuleExpression r)
                {
                        return (m.hasCustomLabel() ? new NamedExpression(m.getLabel(), r) : r);
                }
               
                public RuleExpression visit(SequenceMatcher arg0) {
                        return annotateLabel(arg0, SequenceExpression.of(visitChildren(arg0)));
                }

                public RuleExpression visit(TestMatcher arg0) {
                        return annotateLabel(arg0, UnaryExpression.test(visitChild(arg0)));
                }

                public RuleExpression visit(TestNotMatcher arg0) {
                        return annotateLabel(arg0, UnaryExpression.testNot(visitChild(arg0)));
                }

                public RuleExpression visit(ZeroOrMoreMatcher arg0) {
                        return annotateLabel(arg0, UnaryExpression.zeroOrMore(visitChild(arg0)));
                }
                private String describe(Matcher m) {
                        StringBuilder sb = new StringBuilder();
                        sb.append(m);
                        sb.append(":");
                        if (m.hasCustomLabel())
                                sb.append('c');
                        if (m.isNodeSkipped())
                                sb.append('k');
                        if (m.isNodeSuppressed())
                                sb.append('u');
                        if (m.areSubnodesSuppressed())
                                sb.append('s');
                        return sb.toString();
                }
                private List<RuleExpression> visitChildren(Matcher m)
                {
                        List<RuleExpression> result = new ArrayList<RuleExpression>();
                        for (Matcher mc : m.getChildren())
                        {
                                if (isCustomRule(mc))
                                {
                                        RuleExpression r = new PlaceholderExpression(mc.getLabel());
                                        result.add(r);
                                        maybeDefer(mc);
                                       
                                }
                                else
                                {
                                        RuleExpression r = mc.accept(this);
                                        if (r != null)
                                                result.add(r);
                                }
                        }
                        return result;
                }
                private void maybeDefer(Matcher mc) {
                        boolean doDefer = true;
                        if (mc instanceof StringMatcher)
                                doDefer = false;
                        if (doDefer)
                                defer(mc);
                }

                private RuleExpression visitChild(Matcher m)
                {
                        List<Matcher> mc = m.getChildren();
                        if (mc.size() != 1)
                                throw new RuntimeException("expected 1 item");
                        Matcher mc0 = mc.get(0);
                        if (isCustomRule(mc0))
                        {
                                defer(mc0);
                                return new PlaceholderExpression(mc0.getLabel());
                        }
                        else
                        {
                                return mc0.accept(this);
                        }
                }
               
                private boolean isCustomRule(Matcher mc) {
                        return mc.hasCustomLabel() &&
                         !((mc instanceof ActionMatcher)
                                         | (mc instanceof CharIgnoreCaseMatcher)
                                         | (mc instanceof CharMatcher)
                                         | (mc instanceof CharRangeMatcher)
                                         | (mc instanceof AnyOfMatcher)
                          );
                }
        }

       
        public List<RuleExpression> convert(Matcher rule)
        {
                final Set<Matcher> nodesPrinted = new HashSet<Matcher>();
                final Queue<Matcher> todo = new LinkedList<Matcher>();
                List<RuleExpression> result = new ArrayList<RuleExpression>();
               
                todo.add(rule);
                while (!todo.isEmpty())
                {
                        RuleExpression expr = serviceQueue(nodesPrinted, todo);
                        if (expr != null)
                                result.add(expr);
                }
                return result;
        }

        private RuleExpression serviceQueue(final Set<Matcher> completed, final Queue<Matcher> q) {
                Matcher rule = q.poll();
                if (rule == null)
                        return null;
                if (!completed.add(rule))
                        return null;

                Helper h = new Helper(){
                        @Override public void defer(Matcher m) {
                                q.add(m);
                        }
                };
                RuleExpression re = rule.accept(h);
                return re;
        }
}
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rule reflection

mathias
Administrator
> What are EmptyMatcher and NothingMatcher, and when would they ever be used?

The EmptyMatcher always matches nothing and the NothingMatcher never matches anything...
Or said differently: The EmptyMatcher always succeeds without consuming any input while the NothingMatcher fails on every input.

Most of the time you will not really need them in your ordinary rules, but, for example, whenever you start writing meta-code that creates parboiled rules programmatically it might be nice to have these around.

Cheers,
Mathias

PS: Your grammar printer is interesting. What exactly do you use it for?

---
[hidden email]
http://www.parboiled.org

On 01.03.2011, at 15:06, jms_nh [via parboiled users] wrote:

> thanks, that was a good starting point.
>
> What are EmptyMatcher and NothingMatcher, and when would they ever be used?
>
> just for posterity's sake, I've got a preliminary grammar-printing class (see below, it has a few unfinished loose ends). Working directly on the Matcher rules themselves was awkward, so I wrote a RuleConverter class to transform a graph of Matcher objects to a list of (simpler) RuleExpressions, which can be printed directly via toString(), or visited for custom presentation.
>
> =====
>
> package com.example.parser;
>
> import java.util.ArrayList;
> import java.util.Collections;
> import java.util.HashSet;
> import java.util.LinkedList;
> import java.util.List;
> import java.util.Queue;
> import java.util.Set;
> import org.parboiled.matchers.ActionMatcher;
> import org.parboiled.matchers.AnyMatcher;
> import org.parboiled.matchers.AnyOfMatcher;
> import org.parboiled.matchers.CharIgnoreCaseMatcher;
> import org.parboiled.matchers.CharMatcher;
> import org.parboiled.matchers.CharRangeMatcher;
> import org.parboiled.matchers.CustomMatcher;
> import org.parboiled.matchers.EmptyMatcher;
> import org.parboiled.matchers.FirstOfMatcher;
> import org.parboiled.matchers.Matcher;
> import org.parboiled.matchers.NothingMatcher;
> import org.parboiled.matchers.OneOrMoreMatcher;
> import org.parboiled.matchers.OptionalMatcher;
> import org.parboiled.matchers.SequenceMatcher;
> import org.parboiled.matchers.StringMatcher;
> import org.parboiled.matchers.TestMatcher;
> import org.parboiled.matchers.TestNotMatcher;
> import org.parboiled.matchers.ZeroOrMoreMatcher;
> import org.parboiled.matchervisitors.MatcherVisitor;
>
> public class RuleConverter {
>       final static public ExpressionVisitor<String> stringifier
>               = new ExpressionVisitor<String>()
>       {
>               public String visit(UnaryExpression expr) {
>                       return expr.getModifier().getPrefix()+getSubstring(expr.getSubject())+expr.getModifier().getSuffix();
>               }
>
>               public String visit(FirstOfExpression expr) {
>                       return join(expr.getChildren(), "", " | ", "");
>               }
>
>               public String visit(SequenceExpression expr) {
>                       return join(expr.getChildren(), "{", ", ", "}");
>               }
>
>               public String visit(CharExpression expr) {
>                       return expr.getChars();
>               }
>
>               public String visit(PlaceholderExpression expr) {
>                       return expr.getName();
>               }
>
>               public String visit(NamedExpression expr) {
>                       return expr.getName()+" ::= "+expr.getSubject().accept(this);
>               }
>
>               /* ------------------------------------------------- */
>
>               private String getSubstring(RuleExpression subject) {
>                       String baseString = subject.accept(this);
>                       if (subject instanceof SequenceExpression)
>                               return baseString;
>                       return "("+baseString+")";
>               }
>
>               final protected String join(List<RuleExpression> list, String prefix, String separator, String suffix)
>               {
>                       StringBuilder sb = new StringBuilder();
>                       sb.append(prefix);
>                       int L = list.size();
>                       int i = 0;
>                       for (RuleExpression re : list)
>                       {
>                               sb.append(re.accept(this));
>                               if (++i < L)
>                               {
>                                       sb.append(separator);
>                               }
>                       }
>                       sb.append(suffix);
>                       return sb.toString();
>               }
>
>       };
>
>       public interface RuleExpression
>       {
>               public <T> T accept(ExpressionVisitor<T> visitor);
>       }
>       static abstract public class BaseRuleExpression implements RuleExpression
>       {
>               @Override public String toString() { return accept(stringifier); }
>       }
>       public interface ExpressionVisitor<T>
>       {
>               public T visit(UnaryExpression expr);
>               public T visit(FirstOfExpression expr);
>               public T visit(SequenceExpression expr);
>               public T visit(CharExpression expr);
>               public T visit(PlaceholderExpression expr);
>               public T visit(NamedExpression expr);
>       }
>       enum UnaryModifier {
>               ZERO_OR_MORE("*","","*"),
>               ONE_OR_MORE("+","","+"),
>               OPTIONAL("?","","?"),
>               TEST("&","&",""),
>               TEST_NOT("!","!",""),
>               ;
>
>               final private String sym, pre, post;
>               private UnaryModifier(String sym, String pre, String post) { this.sym = sym; this.pre = pre; this.post = post; }
>               public String getPrefix() { return this.pre; }
>               public String getSuffix() { return this.post; }
>               public String getSymbol() { return this.sym; }
>
>               public UnaryExpression wrap(RuleExpression subject)
>               {
>                       return new UnaryExpression(this, subject);
>               }
>       }
>       static public class UnaryExpression extends BaseRuleExpression
>       {
>               final private UnaryModifier modifier;
>               final private RuleExpression subject;
>
>               public UnaryModifier getModifier() { return this.modifier; }
>               public RuleExpression getSubject() { return this.subject; }
>               private UnaryExpression(UnaryModifier modifier, RuleExpression subject) {
>                       this.modifier = modifier; this.subject = subject;
>               }
>
>               static public UnaryExpression zeroOrMore(RuleExpression subject) { return UnaryModifier.ZERO_OR_MORE.wrap(subject); }
>               static public UnaryExpression optional(RuleExpression subject) { return UnaryModifier.OPTIONAL.wrap(subject); }
>               static public UnaryExpression oneOrMore(RuleExpression subject) { return UnaryModifier.ONE_OR_MORE.wrap(subject); }
>               static public UnaryExpression test(RuleExpression subject) { return UnaryModifier.TEST.wrap(subject); }
>               static public UnaryExpression testNot(RuleExpression subject) { return UnaryModifier.TEST_NOT.wrap(subject); }
>               public String asSubstring() { return "("+toString()+")"; }
>
>               public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
>       }
>
>       abstract static public class ListExpression extends BaseRuleExpression
>       {
>               final private List<RuleExpression> list;
>               public ListExpression(List<RuleExpression> list)
>               {
>                       this.list = Collections.unmodifiableList(new ArrayList<RuleExpression>(list));
>               }
>               public List<RuleExpression> getChildren() { return this.list; }
>       }
>       static public class SequenceExpression extends ListExpression
>       {
>               private SequenceExpression(List<RuleExpression> list) { super(list); }
>               static public RuleExpression of(List<RuleExpression> list)
>               {
>                       if (list.size() == 1)
>                               return list.get(0);
>                       else
>                               return new SequenceExpression(list);
>               }
>               public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
>       }
>       static public class FirstOfExpression extends ListExpression
>       {
>               private FirstOfExpression(List<RuleExpression> list) { super(list); }
>               static public RuleExpression of(List<RuleExpression> list)
>               {
>                       if (list.size() == 1)
>                               return list.get(0);
>                       else
>                               return new FirstOfExpression(list);
>               }
>               public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
>       }
>       static public class CharExpression extends BaseRuleExpression
>       {
>               final private String chars;
>               private CharExpression(String chars) { this.chars = chars; }
>               public String getChars() { return this.chars; }
>               static public CharExpression of(Matcher m)
>               {
>                       return new CharExpression(m.getLabel());
>               }
>               static public CharExpression of(CharRangeMatcher m)
>               {
>                       return new CharExpression("'"+m.cLow+"'..'"+m.cHigh+"'");
>               }
>               public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
>       }
>       static public class PlaceholderExpression extends BaseRuleExpression
>       {
>               final private String name;
>               public PlaceholderExpression(String name) {
>                       this.name = name;
>               }
>               public String getName() { return this.name; }
>               public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
>       }
>       static public class NamedExpression extends PlaceholderExpression
>       {
>               final private RuleExpression subject;
>               public NamedExpression(String name, RuleExpression subject) {
>                       super(name);
>                       this.subject = subject;
>               }
>               public RuleExpression getSubject() { return this.subject; }
>               @Override public <T> T accept(ExpressionVisitor<T> visitor) { return visitor.visit(this); }
>       }
>
>       abstract static class Helper implements MatcherVisitor<RuleExpression> {
>
>               abstract public void defer(Matcher m);
>
>               public RuleExpression visit(ActionMatcher arg0) { return null; }
>               public RuleExpression visit(AnyMatcher arg0) {
>                       System.out.println("any{");
>                       System.out.println("}");
>                       return null;
>               }
>
>               public RuleExpression visit(CharIgnoreCaseMatcher arg0) {
>                       return CharExpression.of(arg0);
>               }
>
>               public RuleExpression visit(CharMatcher arg0) {
>                       return CharExpression.of(arg0);
>               }
>
>               public RuleExpression visit(CustomMatcher arg0) {
>                       System.out.println("custom:"+arg0);
>                       return null;
>               }
>
>               public RuleExpression visit(CharRangeMatcher arg0) {
>                       return CharExpression.of(arg0);
>               }
>
>               public RuleExpression visit(AnyOfMatcher arg0) {
>                       return CharExpression.of(arg0);
>               }
>
>               public RuleExpression visit(EmptyMatcher arg0) {
>                       System.out.println("empty:"+describe(arg0));
>                       return null;
>               }
>
>               public RuleExpression visit(FirstOfMatcher arg0) {
>                       return annotateLabel(arg0, FirstOfExpression.of(visitChildren(arg0)));
>               }
>
>
>               public RuleExpression visit(NothingMatcher arg0) {
>                       System.out.println("nothing{}");
>                       return null;
>               }
>
>               public RuleExpression visit(OneOrMoreMatcher arg0) {
>                       return annotateLabel(arg0, UnaryExpression.oneOrMore(visitChild(arg0)));
>               }
>
>               public RuleExpression visit(OptionalMatcher arg0) {
>                       return annotateLabel(arg0, UnaryExpression.optional(visitChild(arg0)));
>               }
>
>               public RuleExpression annotateLabel(Matcher m, RuleExpression r)
>               {
>                       return (m.hasCustomLabel() ? new NamedExpression(m.getLabel(), r) : r);
>               }
>
>               public RuleExpression visit(SequenceMatcher arg0) {
>                       return annotateLabel(arg0, SequenceExpression.of(visitChildren(arg0)));
>               }
>
>               public RuleExpression visit(TestMatcher arg0) {
>                       return annotateLabel(arg0, UnaryExpression.test(visitChild(arg0)));
>               }
>
>               public RuleExpression visit(TestNotMatcher arg0) {
>                       return annotateLabel(arg0, UnaryExpression.testNot(visitChild(arg0)));
>               }
>
>               public RuleExpression visit(ZeroOrMoreMatcher arg0) {
>                       return annotateLabel(arg0, UnaryExpression.zeroOrMore(visitChild(arg0)));
>               }
>               private String describe(Matcher m) {
>                       StringBuilder sb = new StringBuilder();
>                       sb.append(m);
>                       sb.append(":");
>                       if (m.hasCustomLabel())
>                               sb.append('c');
>                       if (m.isNodeSkipped())
>                               sb.append('k');
>                       if (m.isNodeSuppressed())
>                               sb.append('u');
>                       if (m.areSubnodesSuppressed())
>                               sb.append('s');
>                       return sb.toString();
>               }
>               private List<RuleExpression> visitChildren(Matcher m)
>               {
>                       List<RuleExpression> result = new ArrayList<RuleExpression>();
>                       for (Matcher mc : m.getChildren())
>                       {
>                               if (isCustomRule(mc))
>                               {
>                                       RuleExpression r = new PlaceholderExpression(mc.getLabel());
>                                       result.add(r);
>                                       maybeDefer(mc);
>
>                               }
>                               else
>                               {
>                                       RuleExpression r = mc.accept(this);
>                                       if (r != null)
>                                               result.add(r);
>                               }
>                       }
>                       return result;
>               }
>               private void maybeDefer(Matcher mc) {
>                       boolean doDefer = true;
>                       if (mc instanceof StringMatcher)
>                               doDefer = false;
>                       if (doDefer)
>                               defer(mc);
>               }
>
>               private RuleExpression visitChild(Matcher m)
>               {
>                       List<Matcher> mc = m.getChildren();
>                       if (mc.size() != 1)
>                               throw new RuntimeException("expected 1 item");
>                       Matcher mc0 = mc.get(0);
>                       if (isCustomRule(mc0))
>                       {
>                               defer(mc0);
>                               return new PlaceholderExpression(mc0.getLabel());
>                       }
>                       else
>                       {
>                               return mc0.accept(this);
>                       }
>               }
>
>               private boolean isCustomRule(Matcher mc) {
>                       return mc.hasCustomLabel() &&
>                        !((mc instanceof ActionMatcher)
>                                        | (mc instanceof CharIgnoreCaseMatcher)
>                                        | (mc instanceof CharMatcher)
>                                        | (mc instanceof CharRangeMatcher)
>                                        | (mc instanceof AnyOfMatcher)
>                         );
>               }
>       }
>
>
>       public List<RuleExpression> convert(Matcher rule)
>       {
>               final Set<Matcher> nodesPrinted = new HashSet<Matcher>();
>               final Queue<Matcher> todo = new LinkedList<Matcher>();
>               List<RuleExpression> result = new ArrayList<RuleExpression>();
>
>               todo.add(rule);
>               while (!todo.isEmpty())
>               {
>                       RuleExpression expr = serviceQueue(nodesPrinted, todo);
>                       if (expr != null)
>                               result.add(expr);
>               }
>               return result;
>       }
>
>       private RuleExpression serviceQueue(final Set<Matcher> completed, final Queue<Matcher> q) {
>               Matcher rule = q.poll();
>               if (rule == null)
>                       return null;
>               if (!completed.add(rule))
>                       return null;
>
>               Helper h = new Helper(){
>                       @Override public void defer(Matcher m) {
>                               q.add(m);
>                       }
>               };
>               RuleExpression re = rule.accept(h);
>               return re;
>       }
> }
>
>
> If you reply to this email, your message will be added to the discussion below:
> http://users.parboiled.org/rule-reflection-tp2544785p2603105.html
> To start a new topic under parboiled users, email [hidden email]
> To unsubscribe from parboiled users, click here.

---
[hidden email]
http://www.parboiled.org

---
[hidden email]
http://www.parboiled.org

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rule reflection

jms_nh
mathias wrote
The EmptyMatcher always matches nothing and the NothingMatcher never matches anything...
Or said differently: The EmptyMatcher always succeeds without consuming any input while the NothingMatcher fails on every input.
Ah, thanks! I got a grin out of that one. :-)

>PS: Your grammar printer is interesting. What exactly do you use it for?

I have a script parser in a program I wrote, and a coworker has occasionally been pestering me both to (a) document the syntax, and (b) use a compiler-compiler or something to parse the script (rather than hand-crafted Java + regular expressions which is what the code does currently). I've been looking at ANTLR for a while but it stinks to high heaven if you ask me, so parboiled fits the bill nicely for part (b)... I just needed a way to produce output that is readable by someone familiar with BNF or pseudo-BNF syntax, so that I can just run the grammar printer + don't have to worry about forgetting to update my documentation.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: rule reflection

mathias
Administrator
> I have a script parser in a program I wrote, and a coworker has occasionally been pestering me both to (a) document the syntax, and (b) use a compiler-compiler or something to parse the script (rather than hand-crafted Java + regular expressions which is what the code does currently). I've been looking at ANTLR for a while but it stinks to high heaven if you ask me, so parboiled fits the bill nicely for part (b)... I just needed a way to produce output that is readable by someone familiar with BNF or pseudo-BNF syntax, so that I can just run the grammar printer + don't have to worry about forgetting to update my documentation.

Cool, sounds like an interesting solution.

Let me know, if you have any more questions or feedback wrt. parsing with parboiled.

Cheers,
Mathias
---
[hidden email]
http://www.parboiled.org

On 02.03.2011, at 14:54, jms_nh [via parboiled users] wrote:

> mathias wrote:
> The EmptyMatcher always matches nothing and the NothingMatcher never matches anything...
> Or said differently: The EmptyMatcher always succeeds without consuming any input while the NothingMatcher fails on every input.
> Ah, thanks! I got a grin out of that one. :-)
>
> >PS: Your grammar printer is interesting. What exactly do you use it for?
>
> I have a script parser in a program I wrote, and a coworker has occasionally been pestering me both to (a) document the syntax, and (b) use a compiler-compiler or something to parse the script (rather than hand-crafted Java + regular expressions which is what the code does currently). I've been looking at ANTLR for a while but it stinks to high heaven if you ask me, so parboiled fits the bill nicely for part (b)... I just needed a way to produce output that is readable by someone familiar with BNF or pseudo-BNF syntax, so that I can just run the grammar printer + don't have to worry about forgetting to update my documentation.
>
> If you reply to this email, your message will be added to the discussion below:
> http://users.parboiled.org/rule-reflection-tp2544785p2613421.html
> To start a new topic under parboiled users, email [hidden email]
> To unsubscribe from parboiled users, click here.

Loading...