how to speed up my ArrayList searching?
I currently have an ArrayList
holding objects of a class I have created, I then parse through the ArrayList
in a for loop
searching and comparing some data from the ArrayList
and some global variables
that are loaded else where, however this ArrayList
is constantly growing and will eventually have about 115 elements to it towards the end, which then takes a very long time to search through, the function that does this is also called once for every line I read from a text file and the text file will usually be around 400-500 lines long so as you can tell it is very slow process even when testing on small files. Is there a way to speed this up by maybe using another collection
instead of an ArrayList
, my reasoning for using the ArrayList
is I have to know what index it is on when it finds a match.
Here is the class:
private ArrayList<PanelData> panelArray = new ArrayList<PanelData>(1);
public class PanelData {
String dev = "";
String inst = "";
double tempStart = 0.0;
double tempEnd = 0.0;
}
Function:
public void panelTimeHandler (double timeStart, double timeEnd) throws SQLException {
PanelData temps = new PanelData();
temps.dev = devIDStr;
temps.inst = instanceStr;
temps.tempStart = timeStart;
temps.tempEnd = timeEnd;
boolean flag = false;
if(!flag)
{
panelArray.add(temps);
flag = true;
}
for(int i = 0; i < panelArray.size(); ++i ) {
if(panelArray.get(i).dev.equals(devIDStr) && panelArray.get(i).inst.equals(instanceStr)) {
if(panelArray.get(i).tempStart <= timeStart && panelArray.get(i).tempEnd >= timeEnd ) {
//Do Nothing
}
else
{
temps.dev = devIDStr;
temps.inst = instanceStr;
temps.tempStart = timeStart;
temps.tempEnd = timeEnd;
insert();
panelArray.set(i, temps);
}
}
else
{
temps.dev = devIDStr;
temps.inst = instanceStr;
temps.tempStart = timeStart;
temps.tempEnd = timeEnd;
panelArray.add(temps);
insert();
}
}
}
If there is something more you would like to see just ask, thanks. Beef.
Update: Added insert() function
private void insert() throws SQLException
{
stmt = conn.createStatement();
String sqlStm = "update ARRAY_BAC_SCH_Schedule set SCHEDULE_TIME = {t '" + finalEnd + "'} WHERE SCHEDULE_TIME >= {t '" + finalStart + "'} AND" +
" SCHEDULE_TIME <= {t '" + finalEnd + "'} AND VALUE_ENUM = 0 AND DEV_ID = " + devIDStr + " and INSTANCE = " + instanceStr;
int updateSuccess = stmt.executeUpdate(sqlStm);
if (updateSuccess < 1)
{
sqlStm = "insert into ARRAY_BAC_SCH_Schedule (SITE_ID, DEV_ID, INSTANCE, DAY, SCHEDULE_TIME, VALUE_ENUM, Value_Type) " +
" values (1, " + devIDStr + ", " + instanceStr + ", " + day + ", {t '" + finalStart + "'}, 1, 'Unsupported')";
stmt.executeUpdate(sqlStm);
sqlStm = "insert into ARRAY_BAC_SCH_Schedule (SITE_ID, DEV_ID, INSTANCE, DAY, SCHEDULE_TIME, VALUE_ENUM, Value_Type) " +
" values (1," + devIDStr + ", " + instanceStr + ", " + day + ", {t '" + finalEnd + "'}, 0, 'Unsupported')";
stmt.executeUpdate(sqlStm开发者_运维问答);
}
if(stmt!=null)
stmt.close();
}
Update:
Thank you to Matteo, I realized I was adding to the array even if I didnt find a match till the 10th element it would then added to the array the first 9 times which created many extra elements in the array, which was why it was so slow, I added some breaks and did a little tweaking in the function, and it improved the performance a lot. Thanks for all the input
you can use LinkedHashSet. It seems you add only elements to the end of the list, which is exactly what LinkedHashSet
does as well, when inserting an element.
Note however, a LinkedHashSet
will not allow duplicates, since it is a set.
Searching if an element exists will be O(1) using contains()
Using the LinkedHashSet
will also allow you to keep track of where an element was added, and iterating it will be in order of insertion.
What about using a hashmap
?
I would create a small class for the key:
class Key {
String dev, instr;
// todo: implements equals & hashCode
}
and create the map:
Map<Key, PanelData> map = new HashMap...
then you can easily find the element you need by invoking map.get(new Key(...))
.
Instead of creating a new class, you could also tweak the PanelData class, implementing methods equals & hashcode so that two classes are equal iff their dev
and instr
are equal. In this case, your map becomes:
Map<PanelData, PanelData> map ...
// to add:
map.put(temps, temps)
// to search:
PanelData elem = map.get(new PanelData(desiredDev, desiredInstr));
Quite a few optimiztions here.
1) the call: panelArray.get(i) is used repeatedly. Declare a PanelData variable outside the loop, but initialize it only once, at the very begining of the loop:
PanelData pd = null;
for (int i = 0; i < panelArray.size(); ++i) {
pd = panelArray.get(i);
...
}
2) If your dataset allows it, consider using a few maps to help speed look up times:
HashMap<String, PanelData> devToPanelDataMapping = new HashMap<String,PanelData>();
HashMap<String, PanelData> instToPanelDataMapping = new HashMap<String,PanelData>();
3) Consider hashing your strings into ints or longs since String.equals() is slow compared to (int == int)
4) If the ArrayList will be read only, perhaps a multithread solution may help. The thread that reads lines from the text file can hand out individual lines of data to different 'worker' threads.
1) Create PanelArray with the max expected size + 10% when you first create it.
List<PanelData> panelArray = new ArrayList<PanelData>(130)
- this will prevent dynamic reallocations of the array which will save processing time.
2) What does insert()
do? Odds are that is your resource hog.
This problem might best be solved with a different data structure such as a HashMap
or SortedSet
.
In order to use a HashMap
, you would need to define a class that can produce a hash code for the dev
and inst
string pairs. One solution is something like:
public class DevAndInstPair
{
private String dev, inst;
@Override
public int hashCode() {
return ((dev.hashCode() * 0x490aac18) ^ inst.hashCode());
}
@Override
public boolean equals(Object o) {
if (o == null || !(o instanceof DevAndInstPair)) {
return false;
}
DevAndInstPair other = (DevAndInstPair) o;
return (dev.equals(other.dev) && inst.equals(other.inst));
}
}
You would then use HashMap<DevAndInstPair, PanelData>
as the map type.
Alternatively, if you know that a certain character never appears in dev
strings, then you can use that character as a delimiter separating the dev
value from the inst
value. Supposing that this character is a hyphen ('-'), the key values would be dest + '-' + inst
and the key type of the map would be String
.
To use SortedSet
, you would either have PanelData
implement Comparable<PanelData>
or write a class implementing Comparator<PanelData>
. Remember that the compare operation must be consistent with equals.
A SortedSet
is somewhat trickier to use than a HashMap
, but I personally think that it is the more elegant solution to this problem.
精彩评论