MySQL table: How can I remove specific characters from all fields?
Every week, I have to completely replace the data in several very large MySQL tables. So what I normally do is delete the existing data, import the new data, and then run my usual queries to modify the new data as needed.
Unfortunately, these days I have noticed that the new data contains unwanted characters, such as quotes and extra spaces. With well over 100,000 records in so开发者_如何学Cme of these tables (AFAIK), I cannot easily open the data in notepad to strip out unwanted characters, prior to importing.
I realize I could write a separate find and replace query for every single column in every table, like this:
UPDATE mytablename SET mycolumn = REPLACE(mycolumn, '"', '');
But having to name every column is a bother. Anyway, I would like to find a more elegant solution. Today, I found a snippet on the internet that looks like a start:
SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE
table_name = 'myTable' and ordinal_position = 1
I think the next step might be to loop through the ordinal positions, and then replace and update each column, but I don't know how to do this in MySQL. I also don't know how to stop the loop after the last column is reached, to avoid error messages.
Is there an easy way to do this? Or am I hoping for too much?
I am a beginner, so a clear, simple explanation would be much appreciated.
Thanks in advance.
MORE INFORMATION:
- Since my first post, I have discovered that stored procedures are not allowed on my server. Too bad.
- Anyway, I have tried this new code, just to get started:
set @mytablestring='mytable';
set @mycolumnnumber=1;
set @mycolumnname=(SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = @mytablestring and ordinal_position = @mycolumnnumber);
SELECT @mycolumnname FROM mytable;
Unfortunately, in the final SELECT query, @mycolumnname is interpreted as a string, not as a column name. So the query does not work. If I could get past this, I believe I could write some code to loop through the columns by incrementing @mycolumnnumber.
If anyone knows how to solve this, I would really appreciate some help.
Many thanks.
I suggest that you take a look at vim, sed, awk and many of the other text editors and text processing utilities that you can find on Linux (and sometimes on Windows too). 100,000 records may be a pain in Notepad, but it's a piece of cake for real text processing utilities.
For example, to strip all #
characters from foobar.txt
:
sed 's/#//g' foobar.txt > foobar-clean.txt
Or, the same thing with the file opened in (g)vim:
:%s/#//g
精彩评论