Parsing Permutation Phrases

Transcription

1 Under onsidertion for pulition in J. Funtionl Progrmming 1 F U N C T I O N A L P E A R L Prsing Permuttion Phrses ARTHUR I. BAARS, ANDRES LÖH nd S. DOAITSE SWIERSTRA Institute of Informtion nd Computing Sienes, Utreht University, Utreht, The Netherlnds (e-mil: {rthur,ndres,doitse}@s.uu.nl) Astrt A permuttion phrse is sequene of elements (possily of different types) in whih eh element ours extly one nd the order is irrelevnt. Some of the permutle elements my e optionl. We show how to extend prser omintor lirry with support for prsing suh free-order onstruts. A user of the lirry n esily write prsers for permuttion phrses nd does not need to re out heking nd reordering the reognized elements. Applitions inlude the genertion of prsers for ttriutes of XML tgs nd Hskell s reord syntx. 1 Introdution Prser omintor lirries hve proved to e very fruitful pplition re of funtionl progrmming lnguges: higher-order funtions nd the possiility to define new infix opertors llow prsers to e expressed in onise nd nturl nottion tht losely resemles the syntx of EBNF grmmrs. At the sme time, the user hs the full strtion power of the underlying progrmming lnguge t hnd. Complex, often reurring ptterns n e expressed y defining new omintors. A speifi prsing prolem is the reognition of permuttion phrses. A permuttion phrse is sequene of elements (possily of different types) in whih eh element ours extly one nd the order is irrelevnt. Some of the permutle elements my e optionl. Sine permuttion phrses re not esily expressed y ontext-free grmmr, the usul pproh is to tkle this prolem in two steps: first prse relxed version of the grmmr, then hek whether the reognized elements form permuttion of the expeted elements. This method, however, hs numer of disdvntges. Deling with permuttion of typed vlues is quite umersome, nd the prolem is often voided y enoding the vlues in universl representtion, thus dding n extr level of interprettion. Furthermore, euse of the two steps involved, error messges nnot e produed until lrger prt of the input hs een onsumed, nd speil re hs to e tken to mke them point to the right position in the ode.

2 2 A. I. Brs, A. Löh nd S. D. Swierstr Permuttion phrses hve een proposed y Cmeron (1993) s n extension to EBNF grmmrs, not iming t greter expressive power, ut t more lrity. Cmeron lso presents pseudo-ode lgorithm to prse permuttion phrses with optionl elements effiiently in n impertive setting. It fils, however, to ddress the types of the onstituents. We show how to extend ny existing prser omintor lirry with support for prsing permuttions of typed, potentilly optionl elements. Possile pplitions inlude the implementtion of Hskell s red funtion where it is desirle to prse the fields of dt type with lelled fields in ny order, nd the prsing of XML tgs whih hve lrge sets of potentilly optionl ttriutes tht my our in ny order. For instne, prser for the XHTML imge tg with some of its ttriutes n e written s follows: imgtg = token "<" > token "img" > ttrs < token "/>" where ttrs = permute (Img <$ <> field "sr" uri <> field "lt" string <> optionl (field "longdes" uri) <> optionl (field "height" int) <> optionl (field "width" int) ) The omintor <> is used to seprte prsers for permutle elements, nd <$ <> n e used to pply semnti funtion. Prsers for permuttion phrses hve to e enlosed y ll to permute. Our pproh mkes use of two fetures tht re not provided y ll funtionl progrmming lnguges: existentilly quntified dt types re used to enode reordering informtion tht permutes the reognized elements to nonil order. Additionlly, we utilize lzy evlution to mke the resulting implementtion effiient. The dministrtive prt of prsing permuttion phrses hs qudrti time omplexity in the numer of permutle elements. The size of the ode, however, is liner in the numer of permutle elements. We therefore hoose Hskell s implementtion lnguge. Existentil types re not prt of the Hskell 98 stndrd (Peyton Jones, 2003), ut re supported y severl urrent Hskell implementtions. The pper is orgnized s follows: Setion 2 explins the prser omintors we uild upon. Setion 3 presents the ide of deling with permuttions in terms of permuttion trees nd explins how suh trees re uilt nd onverted into prsers. In Setion 4, we tke rief look t the pplitions mentioned ove: the prsing of dt types with lelled fields nd the prsing of XML ttriute sets. Setion 5 onludes. 2 Prsing using omintor lirries The use of omintor lirry for desriing prsers insted of writing them y hnd or generting them from seprte formlism is well-known tehnique in funtionl progrmming. As result, there re severl exellent lirries round. For this reson we just riefly present the interfe we will ssume in susequent

3 Funtionl perl 3 infixl 3 infixl 4 <> lss Prser p where fil :: p sueed :: p symol :: Chr p Chr ( <>) :: p ( ) p p ( ) :: p p p Fig. 1. Type lss for prser omintors setions of this pper, ut do not go into the detils of the implementtion. We wnt to stress, however, tht our extension is not tied to ny speifi lirry. We mke use of simple interfe (Röjemo, 1995; Swierstr & Duponheel, 1996) tht is prmetrized y the result type of the prsers nd ssumes list of hrters s input. It n esily e implemented y strightforwrd list-of-suesses prsers (Fokker, 1995; Wdler, 1985). Our permuttion prsers hve lso een implemented for more dvned lirries, suh s the fst, error-orreting prser omintors of Swierstr (2001) nd the mondi-style (Hutton & Meijer, 1988) omintor lirry Prse (Leijen, 2001). The prser interfe used here is given s type lss delrtion in Figure 1. The funtion fil represents the prser tht lwys fils, wheres sueed never onsumes ny input nd lwys returns the given result vlue. The prser symol epts solely the given hrter s input. If this hrter is enountered, symol onsumes nd returns this hrter, otherwise it fils. The <> opertor denotes the sequentil omposition of two prsers, where the result of the first prser is pplied to the result of the seond. The opertor expresses hoie etween two prsers. Finlly, the pplition opertor <> $ is prser trnsformer tht n e used to pply semnti funtion to prse result. It n e defined in terms of sueed nd <>. Mny useful omintors n e uilt on top of these si ones. A smll seletion tht we use in this pper is presented in Figure 2. 3 Permuttion prsers We generte the prser for permuttion phrse from n intermedite tree tht n e uilt using set of speil omintors. We first introdue n uxiliry dt type to represent suh permuttion trees, nd we show how to onvert permuttion tree into orresponding prser. We then investigte how permuttion trees n e modified to e le to hndle permuttions ontining optionl elements. After tht, we diret our ttention towrd how permuttion trees n e uilt inrementlly. In the finl susetion, we demonstrte how to del with seprtors in permuttion phrses.

4 4 A. I. Brs, A. Löh nd S. D. Swierstr infixl 4 <>, $ < $, >, < ( $ f $ <>) :: Prser p ( ) p p <> p = sueed f <> p ( $ x $ < ) :: Prser p p p < p = onst x <> $ p ( < ) :: Prser p p p p p < q = onst <> $ p <> q ( >) :: Prser p p p p p > q = flip onst <> $ p <> q prens :: Prser p p p prens p = symol ( > p < symol ) Fig. 2. Some useful prser omintors 3.1 Permuttion trees A permuttion phrse of set of elements n e expnded into n EBNF definition y summing up ll possile permuttions of the elements. Consider for exmple the permuttion phrse of three elements,, nd. Using Cmeron s nottion for permuttion phrses, we n write it s: s ::= Expnding nd susequently left-ftorizing this permuttion phrse gives us the following EBNF prodution rule: s ::= s ::= ( ) ( ) ( ) The left-ftorized prodution rule n e represented y tree s illustrted in Figure 3. Eh pth from the root to lef in the tree represents prtiulr permuttion. Permuttions with ommon prefix shre the sme sutree, hene the numer of hoies in eh node is limited y the numer of permutle elements. Suh permuttion tree is very suitle s intermedite dt struture for permuttion prser. Prsing permuttion phrse oils down to heking whether the input mthes one of the pths in the permuttion tree. If the grmmr (nd thus the permuttion tree) is miguous, lrge prts of the tree might need to e evluted efore it n e deided whih pth must e followed. Therefore, miguous grmmrs should (s lwys) e voided. However, if the miguity in the grmmr stems from optionl elements in the permuttion phrse, the permuttion tree n e modified in simple wy to resolve the miguity. In Setion 3.2, we develop n implementtion for permuttions without optionl elements, nd in Setion 3.3 extend this solution to over optionlity.

5 Funtionl perl 5 Fig. 3. A permuttion tree ontining three elements 3.2 Permuttion trees without optionl elements We introdue dt type Perms for permuttion trees, whih is prmetrized y type onstrutor p (e.g. the prser type) nd result type. dt Perms p = Empty Choie [Brnh p ] dt Brnh p = x. Br (p x) (Perms p (x )) The Empty onstrutor represents the leves of the tree nd the Choie onstrutor the rnhing nodes. A rnh is onstruted using Br nd onsists of n element, represented s prser, plus sutree. As the types of the elements my differ etween rnhes, we hide their types y existentilly quntifying the x in the definition of Br. The sutrees hve n element type tht is different from the type of the originl tree, mking Perms non-regulr dt type. Eh sutree must ontin informtion how to onstrut vlue of the result type from vlue of the element s type. This is hieved y storing in the lef t the end of eh pth funtion tht effetively reorders the elements on the pth to onstrut vlue of the result type. To show tht reordering is determined y the type of the omponents, we will heneforth write the type (or the element type for type onstrutors) of vrile s n index to its nme. The ide tht eh pth in the tree represents the prser for one of the possile permuttions is refleted y the following simple onversion funtion from permuttion trees to prsers. permute :: Prser p Perms p p permute (Empty v ) = sueed v permute (Choie s ) = foldr ( ) fil (mp prs s ) where prs (Br p x t x ) = flip ($) <> $ p x <> permute t x Lzy evlution plys n importnt role here, in tht it ensures tht the permuttion tree is never omputed ompletely. In ft, for permuttion of n elements, just the n tree elements immeditely elow the root of the tree re required to deide

6 6 A. I. Brs, A. Löh nd S. D. Swierstr whih sutree will e used to prse the rest of the permuttion. From then on, only tht sutree ( permuttion tree of size n 1) is relevnt. Iterting this rgument leds to omplexity of only O(n 2 ). There re two potentil prolems here. First, the underlying prser omintor lirry might try to optimize prsing ehviour y evluting different possile pths. This is prolemti euse permuttion trees re so lrge tht the preomputtion is lerly undesirele. If the lirry hs suh fetures, they should e lolly disled for permuttion trees. Seond, repeted prsing of different permuttions might use multiple pths of the tree to e evluted. In prtie, however, the numer of different permuttions tht tully our in the input is smll ompred to the numer of possile permuttions. 3.3 Permuttion trees with optionl elements Optionl elements n e represented y prsers tht n reognize the empty string nd return defult vlue for this element. However, if permute is lled on permuttion tree tht ontins suh prsers, the resulting prser is miguous.????? Fig. 4. The miguous nd the dpted permuttion tree for optionl Consider the left tree in Figure 4, whih ontins ll permuttions of, n optionl nd. Suppose we wnt to reognize. This n e done in three different wys sine the empty n e reognized efore, fter or fter. The three possile prses re shown s dotted pths in the figure. Fortuntely, it is irrelevnt for the result of prse where extly the empty is derived, sine order is not importnt. This llows us to use strtegy similr to the one proposed y Cmeron (1993): prse nonempty onstituents s they re seen nd llow the prser to stop if ll remining elements re optionl. When the prser stops the defult vlues re returned for ll optionl elements tht hve not een reognized. The right tree in Figure 4 depits this strtegy for our three element exmple. The dditionl leves

7 Funtionl perl 7 mrk the positions where the prser is llowed to stop. The string n now e prsed in only one wy. To implement this strtegy we need to e le to determine whether prser n derive the empty string nd split it into its defult vlue nd its non-empty prt, i.e. prser tht ehves the sme exept tht it does not reognize the empty string. Both prts re represented s Mye vlues: the defult prt is Nothing if nd only if the prser nnot reognize the empty string; if the non-empty prt is Nothing, the prser is either sueed (i.e. it just rries semntis) or it is fil. The splitting of prsers is represented y the PrserSplit lss tht is n extension of the norml Prser lss. If the underlying prser omintor lirry nnot e esily dpted to over this extension, one n lterntively introdue dditionl omintors, similr to <> nd <$ <>, nd let the user mrk the optionl elements expliitly (Leijen, 2001). lss Prser p PrserSplit p where split :: p (Mye, Mye (p )) In the solution tht does not del with optionl elements prser for permuttion follows pth from the root of permuttion tree to lef, i.e. n Empty node. In the presene of optionl elements, however, prser my stop in ny node tht stores only optionl elements. We dpt the Perms dt type to inorporte this dditionl informtion. If ll elements stored in tree re optionl then their defult vlues re stored in defults, otherwise defults is Nothing. The prser stored in eh Brnh is not llowed to derive the empty string. We n express the former Empty onstrutor s funtion now. dt Perms p = Choie{defults :: (Mye ), rnhes :: [Brnh p ]} empty x = Choie (Just x) [ ] The funtion permute is strightforwrdly dpted to the generlized dt type: permute :: Prser p Perms p p permute (Choie d s ) = foldr ( ) exit (mp prs s ) where exit = se d of Just v sueed v Nothing fil prs (Br p x t x ) = flip ($) <> $ p x <> permute t x 3.4 Building permuttion tree Permuttion trees re reted y dding the elements of the permuttion one y one to n initilly empty tree. The funtion dd tkes pir onsisting of n optionl defult vlue nd prser tht does not reognize the empty string, nd dds it to

8 8 A. I. Brs, A. Löh nd S. D. Swierstr n existing tree. dd :: (Mye, p ) Perms p ( ) Perms p dd (d, p ) (Choie d s ) = Choie (d p d ) (Br p t : mp ins s ) where ins (Br p x t x ) = Br p x (dd (d, p ) (mpperms flip t x )) Hving defult vlue mens tht the prser desried y the permuttion tree n ept the empty string. Surely, for the onstruted tree, tht is only possile if oth the originl tree hs defult vlue nd d is not Nothing. Then, the new defult vlue n e uilt from the two using funtion pplition. The funtion p does extly this it is funtion pplition lifted to Mye types. p :: Mye ( ) Mye Mye p (Just f ) (Just x) = Just (f x) p = Nothing We n dd new element (d, p ) to permuttion tree y inserting it in ll possile positions to every permuttion tht is lredy in the tree. The funtion dd expliitly onstruts the tree tht represents the permuttion in whih p is the top element. Additionlly, for eh rnh of the originl tree, the top element is left unhnged, nd (d, p ) is inserted everywhere (y reursive ll to dd) in the sutree. Beuse the new element nd the top element of the rnh re now swpped, the funtion resulting from the sutree of the rnh gets its rguments pssed in the wrong order, whih is repired y pplying flip to tht funtion. The funtion mpperms is mpping funtion on permuttion trees. In rnh, f is omposed with the funtion tht results from the sutree. mpperms :: ( ) Perms p Perms p mpperms f (Choie d s) = Choie (fmp f d) (mp (mpbrnh f ) s) mpbrnh :: ( ) Brnh p Brnh p mpbrnh f (Br p x t x ) = Br p x (mpperms (f ) t x ) We now define two opertors for uilding permuttion trees. The first is n opertor tht extends permuttion tree with new element. (<>) :: PrserSplit p Perms p ( ) p Perms p perms <> p = se split p of (Just e, Just ne ) dd (Just e, ne) perms -- optionl element (Nothing, Just ne ) dd (Nothing, ne) perms -- required element (Just e, Nothing) mpperms ($e) perms -- pure semntis (Nothing, Nothing) Choie Nothing [ ] -- fil The seond provides n empty permuttion tree with initil semntis. It hs

9 similr funtionlity s the <> $ for norml prsers. Funtionl perl 9 (<$ <>) :: PrserSplit p ( ) p Perms p f <$ <> p = empty f <> p An exmple with three permutle elements, orresponding to the tree in Figure 3, n now e relized y: permute ((,,) <$ <> int <> hr <> ool) where int, hr, nd ool re prsers for literls of type Int, Chr, nd Bool, respetively. Then ll permuttions of n integer, hrter nd oolen re epted, nd the results of suessful prse re omined using the triple onstrutor (,,), thus yielding vlue of type (Int, Chr, Bool). 3.5 Seprtors Permutle elements re often seprted y symols tht do not rry mening typilly omms or semiolons. Consider extending the three-element exmple to the Hskell tuple syntx: not just the elements, ut lso the prentheses nd the omms should e prsed. Sine there is one seprtion symol less thn there re permutle elements, our urrent vrint of permute nnot hndle this prolem. Therefore we define permutesep s generliztion of permute tht epts n dditionl prser for the seprtor s n rgument. The semntis of the seprtors re ignored for the result. permutesep :: Prser p p Perms p p permutesep sep perm = permutesep (sueed ()) sep perm The funtion permutesep now onverts permuttion tree into prser in lmost the sme wy s the former permute, exept tht efore eh permutle element seprtor is prsed. To prevent tht seprtor is expeted efore the first permutle element, we mke use of the following simple trik. The permutesep funtion expets two extr rguments: the first one will e prsed immeditely efore the first element, nd the seond will e used susequently. Using sueed () s first extr rgument in permutesep leds to the desired result. permutesep :: Prser p p p Perms p p permutesep fsep sep (Choie d s ) = foldr ( ) exit (mp prs s ) where exit = se d of Just v sueed v Nothing fil prs (Br p x t x ) = flip ($) < $ fsep <> p x <> permutesep sep sep t x The permute funtion n now e implemented in terms of permutesep. permute :: Prser p Perms p p permute = permutesep (sueed ()) To return to the smll exmple, triples of n integer, hrter, nd oolen

10 10 A. I. Brs, A. Löh nd S. D. Swierstr in ny order re prsed y: prens (permutesep (symol, ) ((,,) <$ <> int <> hr <> ool)) 4 Applitions 4.1 XML ttriutes We now demonstrte the use of the permuttion prsers y showing how to prse XML tgs with ttriutes. For simpliity, we just onsider one tg (the img tg of XHTML) nd only del with suset of the ttriutes llowed. In Hskell progrm, this tg might e represented y the following dt type. dt XHTML = Img{sr :: URI, lt :: String, longdes :: Mye URI, height :: Mye Int, width :: Mye Int }... Our vrint of the img tg hs five ttriutes of three different types. We use Hskell s reord syntx to keep trk of the nmes. The first two ttriutes re mndtory wheres the others re optionl. We hoose the Mye vrint of their types to reflet this optionlity. Our prser should e le to prse the ttriutes in ny order, where ny of the optionl rguments my e omitted. For the prsing proess, we ignore whitespe nd ssume tht there is prser token :: Prser p String p String tht onsumes just the given token nd fils on ny other input. Using the permute omintor, writing the prser for the img tg is esy: imgtg :: PrserSplit p p XHTML imgtg = token "<" > token "img" > ttrs < token "/>" where ttrs = permute $ Img <$ <> field "sr" uri <> field "lt" string <> optionl (field "longdes" uri) <> optionl (field "height" int) <> optionl (field "width" int) optionl :: Prser p p p (Mye ) optionl p = Just <> $ p sueed Nothing The order in whih we denote the ttriutes determines the order in whih the results re returned. Therefore, we n pply the Img onstrutor to form vlue of the XHTML dt type. The helper funtion field is used to prse single ttriute. field :: Prser p String p p field s p = token s > symol = > p

11 Funtionl perl Hskell s reord syntx Hskell llows dt types to ontin lelled fields. If one wnts to onstrut vlue of tht dt type, one n mke use of these nmes. The dvntge is tht the user does not need to rememer the order in whih the fields of the onstrutor hve een defined. Furthermore, ll fields re onsidered s optionl. If field is not expliitly set to vlue, it is silently ssumed to e. Wheres ompilers support order-free syntx (the reord fields in progrm n e ordered ritrrily), the red funtion expets the field in the sme order s in the dt type delrtion. The resulting symmetry is unfortunte. Using the permutesep omintor, it is n esy tsk to write more flexile prsers for dt types with lelled fields. justornothing :: Prser p p p (Mye ) justornothing p = Just < $ token "Just" <> p Nothing < $ token "Nothing" img :: PrserSplit p p XHTML img = token "Img" > token "{" > fields < token "}" where fields = permutesep (symol, ) $ Img <$ <> reordfield "sr" uri <> reordfield "lt" string <> reordfield "longdes" (justornothing uri) <> reordfield "height" (justornothing int) <> reordfield "width" (justornothing int) We use reordfield here to prse single optionl reord field, returning if it is not present. reordfield :: Prser p String p p reordfield f p = field f p sueed 5 Conlusion We hve shown how to extend prser omintor lirry with the funtionlity to prse free-order onstruts. It n e pled on top of ny omintor lirry tht implements the Prser interfe. A user of the lirry n esily write prsers for free-order onstruts nd does not need to re out heking nd reordering the prsed elements. Due to the use of existentilly quntified types the implementtion of reordering is type sfe nd hidden from the user. The underlying prser omintors n e used to hndle errors, suh s missing or duplite elements, sine the extension inherits their error-reporting or errorrepiring properties. We hve shown how our extension n e used to prse XML ttriutes nd Hskell reords. Other interesting exmples mentioned y Cmeron (1993) inlude ittion fields in BiT E X iliogrphies nd ttriute speifiers in C delrtions. Cmeron s pseudo-ode lgorithm uses similr strtegy. It does not show, however, how to mintin type sfety y undoing the hnge in semntis resulting

12 12 A. I. Brs, A. Löh nd S. D. Swierstr from reordering, nor n it del with the presene of seprtors etween free-order onstituents. Referenes Cmeron, R. D. (1993) Extending ontext-free grmmrs with permuttion phrses. ACM Letters on Progrmming Lnguges nd Systems 2(4): Fokker, J. (1995) Funtionl prsers. Advned Funtionl Progrmming, First Interntionl Spring Shool. LNCS 925, pp Springer-Verlg. Hutton, G. nd Meijer, H. (1988) Mondi prser omintors. Journl of Funtionl Progrmming 8(4): Leijen, D. (2001) Prse, fst omintor prser. html. Peyton Jones, S. (2003) Hskell 98 Lnguge nd Lirries. Cmridge University Press. Röjemo, N. (1995) Grge olletion nd memory effiieny in lzy funtionl lnguges. PhD thesis, Chlmers University of Tehnology. Swierstr, D. (2001) Comintor prsers: From toys to tools. Hutton, G. (ed), Eletroni Notes in Theoretil Computer Siene, vol. 41. Elsevier Siene Pulishers. Swierstr, D. nd Duponheel, L. (1996) Deterministi, error orreting omintor prsers. Advned Funtionl Progrmming, Seond Interntionl Spring Shool. LNCS 1129, pp Springer-Verlg. Wdler, P. (1985) How to reple filure with list of suesses. Funtionl Progrmming Lnguges nd Computer Arhiteture. LNCS 201, pp Springer-Verlg.