String literal recognition problem
I'm trying to recognize string literal by reading string per symbol. Example of my scanner skeleton:
public sealed class Scanner
{
// some class inner implementations
/// <summary>
///
/// </summary>
/// <param name="Line"></param>
/// <param name="LineNumber"></param>
public void Run(String Line, Int32 LineNumber)
{
var ChPosition = default(Int32);
var ChCurrent = default(Char);
var Value = new StringBuilder();
while (default(Char) != Line.ElementAtOrDefault<Char>(ChPosition))
{
ChCurrent = Line.ElementAtOrDefault<Char>(ChPosition);
#region [Whitespace]
if (Char.IsWhiteSpace(ChCurrent))
{
ChPosition++;
}
#endregion
else
{
switch (ChCurrent)
{
#region [Str开发者_如何学Cing Literal (")]
case '"':
{
// skipping " sign, include only string inner value
ChCurrent = Line.ElementAtOrDefault<Char>(++ChPosition);
// ...? Problematic place!!!
this.Tokens.Enqueue(new SharedEntities.Token
{
Class = SharedEntities.Token.TokenClass.StringLiteral,
Value = Value.ToString()
}
);
Value.Clear();
ChPosition++;
break;
}
#endregion
{
throw new ScanningException(
"<syntax_error#" + ChCurrent.ToString() + ">\n"
+ "Unsupported character appeared at: {ln: "
+ LineNumber.ToString()
+ "; pos: "
+ (ChPosition + 1).ToString()
+ "}"
);
}
} // [switch(ChCurrent)]
} // [if(Char.IsWhiteSpace(ChCurrent))...else]
} // [while(default(Char) != Line.ElementAtOrDefault<Char>(ChPosition))]
} // [public void Run(String Line, Int32 LineNumber)]
} // [public sealed class Scanner]
My target is to parse pascal-like string: "{everything enclosed, but ", only "" pair is allowed}".
First, you are obviously using some kind of parsing library, you would have better chance if you had modified your code, e.g. to something like I did, so that anybody can copy, paste, run your code.
Answer is simple, your (string literal)-parsing region does not parse all input. Here is your code modified to be used without any additional library:
public class Test
{
static char ElementAtOrDefault(string value, int position)
{
return position >= value.Length ? default(char) : value[position];
}
static string parseStringLiteral(string value, ref int ChPosition)
{
StringBuilder Value = new StringBuilder();
char ChCurrent = ElementAtOrDefault(value, ++ChPosition);
while (ChCurrent != '"')
{
Value.Append(ChCurrent);
ChCurrent = ElementAtOrDefault(value, ++ChPosition);
if (ChCurrent == '"')
{
// "" sequence only acceptable
if (ElementAtOrDefault(value, ChPosition + 1) == '"')
{
Value.Append(ChCurrent);
// skip 2nd double quote
ChPosition++;
// move position next
ChCurrent = ElementAtOrDefault(value, ++ChPosition);
}
}
else if (default(Char) == ChCurrent)
{
// message: unterminated string
throw new Exception("ScanningException");
}
}
ChPosition++;
return Value.ToString();
}
public static void test(string literal)
{
Console.WriteLine("testing literal with " + literal.Length +
" chars:\n" + literal);
try
{
int pos = 0;
string res = parseStringLiteral(literal, ref pos);
Console.WriteLine("Parsed " + res.Length + " chars:\n" + res);
}
catch (Exception ex)
{
Console.WriteLine("Error: " + ex.Message);
}
Console.WriteLine();
}
public static int Main(string[] args)
{
test(@"""Hello Language Design""");
test(@"""Is there any problems with the """"strings""""?""");
test(@"""v#:';?325;.<>,|+_)""(*&^%$#@![]{}\|-_=""");
return 0;
}
}
Running this program produces output:
testing literal with 23 chars: "Hello Language Design" Parsed 21 chars: Hello Language Design testing literal with 45 chars: "Is there any problems with the ""strings""?" Parsed 41 chars: Is there any problems with the "strings"? testing literal with 39 chars: "v#:';?325;.,|+_)"(*&^%$#@![]{}\|-_=" Parsed 18 chars: v#:';?325;.,|+_)
So it works for your testing, but algorithm is not correct, try running:
//literal with "", should produce ", but it does not
test(@"""""""""");
And you will incorrectly get:
testing literal with 4 chars: """" Parsed 0 chars:
Problem is, if you encounter character " in your while condition, you do not check next character, if it is " or not:
while (ChCurrent != '"') //bug
Of course, I created correct version for you :-) Here it is (it uses your style, just edited version of yours):
static string parseStringLiteral(string value, ref int ChPosition)
{
StringBuilder Value = new StringBuilder();
char ChCurrent = ElementAtOrDefault(value, ++ChPosition);
bool goon = true;
while (goon)
{
if (ChCurrent == '"')
{
// "" sequence only acceptable
if (ElementAtOrDefault(value, ChPosition + 1) == '"')
{
Value.Append(ChCurrent);
// skip 2nd double quote
ChPosition++;
// move position next
ChCurrent = ElementAtOrDefault(value, ++ChPosition);
}
else goon = false; //break;
}
else if (default(Char) == ChCurrent)
{
// message: unterminated string
throw new Exception("ScanningException");
}
else
{
Value.Append(ChCurrent);
ChCurrent = ElementAtOrDefault(value, ++ChPosition);
}
}
ChPosition++;
return Value.ToString();
}
Happy coding :-)
精彩评论