Treemap Problem
I am trying to count frequency of words in a text file. But I have to use a different approach. For example, if the file contains BRAIN-ISCHEMIA and ISCHEMIA-BRAIN, I need to count BRAIN-ISCHEMIA twice (and leaving ISCHEMIA-BRAIN) or vice versa. Here is my piece of code-
// Mapping of String->Integer (word -> frequency)
HashMap<String, Integer> frequencyMap = new HashMap<String, Integer>();
// Iterate through each line of the file
String[] temp;
String currentLine;
String currentLine2;
while ((currentLine = in.readLine()) != null) {
// Remove this line if you want words to be case sensitive
currentLine = currentLine.toLowerCase();
temp=currentLine.split("-");
currentLine2=temp[1]+"-"+temp[0];
// Iterate through each word of the current line
// Delimit words based on whitespace, punctuation, and quotes
StringTokenizer parser = new StringTokenizer(currentLine);
while (parser.hasMoreTokens()) {
String currentWord = parser.nextToken();
Integer frequency = frequencyMap.get(currentWord);
// Add the word if it doesn't already exist, otherwise increment the
// frequency counter.
if (frequency == null) {
frequency = 0;
}
frequencyMap.put(currentWord, frequency + 1);
}
StringTokenizer parser2 = new StringTokenizer(currentLine2);
while (parser2.hasMoreTokens()) {
String currentWord2 = parser2.nextToken();
Integer frequency = frequencyMap.get(currentWord2);
// Add the word if it doesn't already exist, otherwise increment the
// frequency counter.
if (frequency == null) {
frequency = 0;
}
frequencyMap.put(currentWord2, frequency + 1);
}
}
// Display our nice little Map
System.out.println(frequencyMap);
But for the following file-
ISCHEMIA-GLUTAMATE ISCHEMIA-BRAIN GLUTAMATE-BRAIN BRAIN-TOLERATE BRAIN-TOLERATE TOLERATE-BRAIN GLUTAMATE-ISCHEMIA ISCHEMIA-GLUTAMATE
I am getting the follow开发者_运维百科ing output-
{glutamate-brain=1, ischemia-glutamate=3, ischemia-brain=1, glutamate-ischemia=3, brain-tolerate=3, brain-ischemia=1, tolerate-brain=3, brain-glutamate=1}
The problem is in second while block I think. Any light on this problem will be highly appreciated.
From an algorithm perspective, you may want to consider the following approach:
For each string, split, then sort, then re-combine (i.e. take DEF-ABC and convert to ABC-DEF. ABC-DEF would convert to ABC-DEF). Then use that as the key for your frequency count.
If you need to hold onto the exact original item, just include that in your key - so the key would have: ordinal (the re-combined string) and original.
Disclaimer: I stole the sweet trick suggested by Kevin Day for my implementation.
I still want to post just to let you know that using the right data structure (Multiset/Bad) and the right libraries (google-guava) will not only simplify the code but also makes it efficient.
Code
public class BasicFrequencyCalculator
{
public static void main(final String[] args) throws IOException
{
@SuppressWarnings("unchecked")
Multiset<Word> frequency = Files.readLines(new File("c:/2.txt"), Charsets.ISO_8859_1, new LineProcessor() {
private final Multiset<Word> result = HashMultiset.create();
@Override
public Object getResult()
{
return result;
}
@Override
public boolean processLine(final String line) throws IOException
{
result.add(new Word(line));
return true;
}
});
for (Word w : frequency.elementSet())
{
System.out.println(w.getOriginal() + " = " + frequency.count(w));
}
}
}
public class Word
{
private final String key;
private final String original;
public Word(final String orig)
{
this.original = orig.trim();
String[] temp = original.toLowerCase().split("-");
Arrays.sort(temp);
key = temp[0] + "-"+temp[1];
}
@Override
public int hashCode()
{
final int prime = 31;
int result = 1;
result = prime * result + ((getKey() == null) ? 0 : getKey().hashCode());
return result;
}
@Override
public boolean equals(final Object obj)
{
if (this == obj)
{
return true;
}
if (obj == null)
{
return false;
}
if (!(obj instanceof Word))
{
return false;
}
Word other = (Word) obj;
if (getKey() == null)
{
if (other.getKey() != null)
{
return false;
}
}
else if (!getKey().equals(other.getKey()))
{
return false;
}
return true;
}
@Override
public String toString()
{
return getOriginal();
}
public String getKey()
{
return key;
}
public String getOriginal()
{
return original;
}
}
Output
BRAIN-TOLERATE = 3
ISCHEMIA-GLUTAMATE = 3
GLUTAMATE-BRAIN = 1
ISCHEMIA-BRAIN = 1
Thanks everyone for your help. Here is how I solved it-
// Mapping of String->Integer (word -> frequency)
TreeMap<String, Integer> frequencyMap = new TreeMap<String, Integer>();
// Iterate through each line of the file
String[] temp;
String currentLine;
String currentLine2;
while ((currentLine = in.readLine()) != null) {
temp=currentLine.split("-");
currentLine2=temp[1]+"-"+temp[0];
// Iterate through each word of the current line
StringTokenizer parser = new StringTokenizer(currentLine);
while (parser.hasMoreTokens()) {
String currentWord = parser.nextToken();
Integer frequency = frequencyMap.get(currentWord);
Integer frequency2 = frequencyMap.get(currentLine2);
// Add the word if it doesn't already exist, otherwise increment the
// frequency counter.
if (frequency == null) {
if (frequency2 == null)
frequency = 0;
else {
frequencyMap.put(currentLine2, frequency2 + 1);
break;
}//else
} //if (frequency == null)
frequencyMap.put(currentWord, frequency + 1);
}//while (parser.hasMoreTokens())
}//while ((currentLine = in.readLine()) != null)
// Display our nice little Map
System.out.println(frequencyMap);
精彩评论