Using ruby's treetop peg to parse a debian Packages.gz
I'm trying to break open a Packages.gz using Ruby's treetop and I've having trouble making keywords and values unambiguous. Here's my treetop grammar:
grammar Debian
rule collection
entry+
end
rule entry
(tag space value)
end
rule package_details
tag value &[^$]
end
rule tag
[A-Za-z0-9\-]+ ":"
end
rule value
(!tag value_line+ "\n")+
end
rule value_line
([A-Za-z0-9 <>@()=\.\-|/,_"':])+
end
rule space
[ \t]+
end
end
And here's my sample input:
Package: acct
Priority: optional
Section: admin
Installed-Size: 352
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Mathieu Trudel <mathieu.tl@gmail.com>
Architecture: i386
Version: 6.5.4-2ubuntu1
Depends: dpkg (>= 1.15.4) | install-info, libc6 (>= 2.4)
Filename: pool/main/a/acct/acct_6.5.4-2ubuntu1_i386.deb
Size: 111226
MD5sum: 10cba1458ace8c31169c0e9e915c9a0f
SHA1: 6c2dcdc480144a9922329cd4fa22c7d1cb83fcbb
SHA256: bf8d8bb8eef3939786a1cefc39f94079f43464b71099f4a59b61b24cafdbc010
Description: The GNU Accounting utilities for process and login accounting
GNU Accounting Utilities is a set of utilities which reports and summarizes
data about user connect times and process execution statistics.
.
"Login accounting" provides summaries of system resource usage based on connect
time, and "process accounting" provides summaries based on the commands
executed on the system.
.
The 'last' command is provided by the sysvinit package and not included here.
Homepage: http://www.gnu.org/software/acct/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 18m
This works almost 100%, but then it fails when examining a url. The problem is that a URL contains a ":" which I don't seem to be able to snag. When I edit the sample's Homepage entry and substitue a "_" for the ":" it goes right through.
This is my first PEG grammar but I can tell that I need to make it less ambiguous/more succinct. Looking over the advanced documentation I would like to define a tag as
rule tag
!(!'\n' .) [A-Za-z0-9\-]+ ":"
end
But I don't understand completely what it is doing. The tag must not(not have a new line OR anything) which I take to mean, (either a newline or nothing). The subtleties escape me...
Would switching to that format help me out?开发者_StackOverflow中文版 Does someone know why that doesn't match?
I appear to have gotten one working grammar at this point:
grammar Debian
# The file is too big for us to emit a package_list. Look at parser.rb to see how I just split the string.
#rule package_list
# (package "\n"?)+ <DebianSyntaxNode::PackageList>
#end
rule package
(tag / value)+ <DebianSyntaxNode::Package>
end
rule tag
tag_value tag_stop <DebianSyntaxNode::Tag>
end
rule tag_value
[\w\-]+ <DebianSyntaxNode::TagValue>
end
rule tag_stop
": " <DebianSyntaxNode::TagStop>
end
rule value
value_line value_stop <DebianSyntaxNode::Value>
# value_line value_stop <DebianSyntaxNode::Value>
end
rule value_line
(!"\n" .)+ <DebianSyntaxNode::ValueLine>
# ([\w \. " , \- ' : / < > @ ( ) = | \[ \] + ; ~ í á * % `])+ <DebianSyntaxNode::ValueLine>
end
rule value_stop
"\n"? <DebianSyntaxNode::ValueStop>
end
end
Problem is that now the value_line doesn't include the "\n" when it's a multiline entry. Plus I have to combine multi-line entries in the parser.
If you want to see where this code is going, check out the little github project I started: https://github.com/derdewey/Debian-Packages-Parser
精彩评论