How to strip all non alphanumeric characters from a string in c++?

I am writing a piece of software, and It require me to handle data I get from a webpage with libcurl. When I get the da开发者_运维知识库ta, for some reason it has extra line breaks in it. I need to figure out a way to only allow letters, numbers, and spaces. And remove everything else, including line breaks. Is there any easy way to do this? Thanks.

Write a function that takes a char and returns true if you want to remove that character or false if you want to keep it:

bool my_predicate(char c);

Then use the std::remove_if algorithm to remove the unwanted characters from the string:

std::string s = "my data";
s.erase(std::remove_if(s.begin(), s.end(), my_predicate), s.end());

Depending on your requirements, you may be able to use one of the Standard Library predicates, like std::isalnum, instead of writing your own predicate (you said you needed to match alphanumeric characters and spaces, so perhaps this doesn't exactly fit what you need).

If you want to use the Standard Library std::isalnum function, you will need a cast to disambiguate between the std::isalnum function in the C Standard Library header <cctype> (which is the one you want to use) and the std::isalnum in the C++ Standard Library header <locale> (which is not the one you want to use, unless you want to perform locale-specific string processing):

s.erase(std::remove_if(s.begin(), s.end(), (int(*)(int))std::isalnum), s.end());

This works equally well with any of the sequence containers (including std::string, std::vector and std::deque). This idiom is commonly referred to as the "erase/remove" idiom. The std::remove_if algorithm will also work with ordinary arrays. The std::remove_if makes only a single pass over the sequence, so it has linear time complexity.

Previous uses of std::isalnum won't compile with std::ptr_fun without passing the unary argument is requires, hence this solution with a lambda function should encapsulate the correct answer:

s.erase(std::remove_if(s.begin(), s.end(), 
[]( auto const& c ) -> bool { return !std::isalnum(c); } ), s.end());

You could always loop through and just erase all non alphanumeric characters if you're using string.

#include <cctype>

size_t i = 0;
size_t len = str.length();
while(i < len){
    if (!isalnum(str[i]) || str[i] == ' '){
        str.erase(i,1);
        len--;
    }else
        i++;
}

Someone better with the Standard Lib can probably do this without a loop.

If you're using just a char buffer, you can loop through and if a character is not alphanumeric, shift all the characters after it backwards one (to overwrite the offending character):

#include <cctype>

size_t buflen = something;
for (size_t i = 0; i < buflen; ++i)
    if (!isalnum(buf[i]) || buf[i] != ' ')
        memcpy(buf[i], buf[i + 1], --buflen - i);

Just extending James McNellis's code a little bit more. His function is deleting alnum characters instead of non-alnum ones.

To delete non-alnum characters from a string. (alnum = alphabetical or numeric)

• Declare a function (isalnum returns 0 if passed char is not alnum)

  bool isNotAlnum(char c) {
      return isalnum(c) == 0;
  }
  

• And then write this

  s.erase(remove_if(s.begin(), s.end(), isNotAlnum), s.end());
  

then your string is only with alnum characters.

Benchmarking the different methods.

If you are looking for a benchmark I made one.

(115830 cycles) 115.8ms -> using stringstream
( 40434 cycles)  40.4ms -> s.erase(std::remove_if(s.begin(), s.end(), [](char c) { return !isalnum(c); }), s.end());
( 40389 cycles)  40.4ms -> s.erase(std::remove_if(s.begin(), s.end(), [](char c) { return ispunct(c); }), s.end());
( 42386 cycles)  42.4ms -> s.erase(remove_if(s.begin(), s.end(), not1(ptr_fun( (int(*)(int))isalnum ))), s.end());
( 42969 cycles)  43.0ms -> s.erase(remove_if(s.begin(), s.end(), []( auto const& c ) -> bool { return !isalnum(c); } ), s.end());
( 44829 cycles)  44.8ms -> alnum_from_libc(s) see below
( 24505 cycles)  24.5ms -> Puzzled? My method, see below
(  9717 cycles)   9.7ms -> using mask and bitwise operators

Original length: 8286208, current len with alnum only: 5822471

• Stringstream gives terrible results (but we all know that)
• The different answers already given gives about the same runtime
• Doing it the C way consistently give better runtime (almost twice faster!), it is definitely worth considering, and on top of that it is compatible with C language.
• My bitwise method (also C compatible) is more than 400% faster.

NB the selected answer had to be modified as it was keeping only the special characters

NB2: The test file is a (almost) 8192 kb text file with roughly 62 alnum and 12 special characters, randomly and evenly written.

Benchmark source code

#include <ctime>

#include <iostream>
#include <sstream>
#include <string>
#include <algorithm>

#include <locale> // ispunct
#include <cctype>

#include <fstream> // read file
#include <streambuf>

#include <sys/stat.h> // check if file exist
#include <cstring>

using namespace std;

bool exist(const char *name)
{
  struct stat   buffer;
  return !stat(name, &buffer);
}

constexpr int SIZE = 8092 * 1024;

void keep_alnum(string &s) {
    stringstream ss;
    int i = 0;
    for (i = 0; i < SIZE; i++)
        if (isalnum(s[i]))
            ss << s[i];
    s = ss.str();
}

/* my method, best runtime */
void old_school(char *s) {
    int n = 0;
    for (int i = 0; i < SIZE; i++) {
        unsigned char c = s[i] - 0x30; // '0'
        if (c < 10 || (c -= 0x11) < 26 || (c -= 0x20) < 26) // 0x30 + 0x11 = 'A' + 0x20 = 'a'
            s[n++] = s[i];
    }
    s[n] = '\0';
}

void alnum_from_libc(char *s) {
    int n = 0;
    for (int i = 0; i < SIZE; i++) {
        if (isalnum(s[i]))
            s[n++] = s[i];
    }
    s[n] = '\0';
}

#define benchmark(x) printf("\033[30m(%6.0lf cycles) \033[32m%5.1lfms\n\033[0m", x, x / (CLOCKS_PER_SEC / 1000))

int main(int ac, char **av) {
    if (ac < 2) {
        cout << "usage: ./a.out \"{your string} or ./a.out FILE \"{your file}\n";
        return 1;
    }
    string s;
    s.reserve(SIZE+1);
    string s1;
    s1.reserve(SIZE+1);
    char s4[SIZE + 1], s5[SIZE + 1];
    if (ac == 3) { 
        if (!exist(av[2])) {
            for (size_t i = 0; i < SIZE; i++)
              s4[i] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnoporstuvwxyz!@#$%^&*()__+:\"<>?,./'"[rand() % 74];
          s4[SIZE] = '\0';
          ofstream ofs(av[2]);
          if (ofs)
            ofs << s4;
        }
        ifstream ifs(av[2]);
        if (ifs) {
          ifs.rdbuf()->pubsetbuf(s4, SIZE);
          copy(istreambuf_iterator<char>(ifs), {}, s.begin());
        }
        else
          cout << "error\n";

        ifs.seekg(0, ios::beg);
        s.assign((istreambuf_iterator<char>(ifs)), istreambuf_iterator<char>());
    }
    else
        s = av[1];

    double elapsedTime;
    clock_t start;
    bool is_different = false;

    s1 = s;
    start = clock();
    keep_alnum(s1);
    elapsedTime = (clock() - start);
    benchmark(elapsedTime);
    string tmp = s1;

    s1 = s;
    start = clock();
    s1.erase(std::remove_if(s1.begin(), s1.end(), [](char c) { return !isalnum(c); }), s1.end());
    elapsedTime = (clock() - start);
    benchmark(elapsedTime);
    is_different |= !!strcmp(tmp.c_str(), s1.c_str());

    s1 = s;
    start = clock();
    s1.erase(std::remove_if(s1.begin(), s1.end(), [](char c) { return ispunct(c); }), s1.end());
    elapsedTime = (clock() - start);
    benchmark(elapsedTime);
    is_different |= !!strcmp(tmp.c_str(), s1.c_str());

    s1 = s;
    start = clock();
    s1.erase(remove_if(s1.begin(), s1.end(), not1(ptr_fun( (int(*)(int))isalnum ))), s1.end());
    elapsedTime = (clock() - start);
    benchmark(elapsedTime);
    is_different |= !!strcmp(tmp.c_str(), s1.c_str());

    s1 = s;
    start = clock();
    s1.erase(remove_if(s1.begin(), s1.end(), []( auto const& c ) -> bool { return !isalnum(c); } ), s1.end());
    elapsedTime = (clock() - start);
    benchmark(elapsedTime);
    is_different |= !!strcmp(tmp.c_str(), s1.c_str());

    memcpy(s4, s.c_str(), SIZE);
    start = clock();
    alnum_from_libc(s4);
    elapsedTime = (clock() - start);
    benchmark(elapsedTime);
    is_different |= !!strcmp(tmp.c_str(), s4);

    memcpy(s4, s.c_str(), SIZE);
    start = clock();
    old_school(s4);
    elapsedTime = (clock() - start);
    benchmark(elapsedTime);
    is_different |= !!strcmp(tmp.c_str(), s4);

    cout << "Original length: " << s.size() << ", current len with alnum only: " << strlen(s4) << endl;
    // make sure that strings are equivalent
    printf("\033[3%cm%s\n", ('3' + !is_different), !is_different ? "OK" : "KO");

    return 0;
}

My solution

For the bitwise method you can check it directly on my github, basically I avoid branching instructions (if) thanks to the mask. I avoid posting bitwise operations with C++ tag, I get a lot of hate for it.

For the C style one, I iterate over the string and have two index: n for the characters we keep and i to go through the string, where we test one after another if it is a digit, a uppercase or a lowercase.

Add this function:

void strip_special_chars(char *s) {
    int n = 0;
    for (int i = 0; i < SIZE; i++) {
        unsigned char c = s[i] - 0x30;
        if (c < 10 || (c -= 0x11) < 26 || (c -= 0x20) < 26) // 0x30 + 0x11 = 'A' + 0x20 = 'a'
            s[n++] = s[i];
    }
    s[n] = '\0';
}

and use as:

char s1[s.size() + 1]
memcpy(s1, s.c_str(), s.size());
strip_special_chars(s1);

The remove_copy_if standard algorithm would be very appropriate for your case.

#include <cctype>
#include <string>
#include <functional>

std::string s = "Hello World!";
s.erase(std::remove_if(s.begin(), s.end(),
    std::not1(std::ptr_fun(std::isalnum)), s.end()), s.end());
std::cout << s << std::endl;

Results in:

"HelloWorld"

You use isalnum to determine whether or not each character is alpha numeric, then use ptr_fun to pass the function to not1 which NOTs the returned value, leaving you with only the alphanumeric stuff you want.

You can use the remove-erase algorithm this way -

// Removes all punctuation       
s.erase( std::remove_if(s.begin(), s.end(), &ispunct), s.end());

Below code should work just fine for given string s. It's utilizing <algorithm> and <locale> libraries.

std::string s("He!!llo  Wo,@rld! 12 453");
s.erase(std::remove_if(s.begin(), s.end(), [](char c) { return !std::isalnum(c); }), s.end());

The mentioned solution

s.erase( std::remove_if(s.begin(), s.end(), &std::ispunct), s.end());

is very nice, but unfortunately doesn't work with characters like 'Ñ' in Visual Studio (debug mode), because of this line:

_ASSERTE((unsigned)(c + 1) <= 256)

in isctype.c

So, I would recommend something like this:

inline int my_ispunct( int ch )
{
    return std::ispunct(unsigned char(ch));
}
...
s.erase( std::remove_if(s.begin(), s.end(), &my_ispunct), s.end());

The following works for me.

str.erase(std::remove_if(str.begin(), str.end(), &ispunct), str.end());
str.erase(std::remove_if(str.begin(), str.end(), &isspace), str.end());

void remove_spaces(string data)
{ int i=0,j=0;
    while(i<data.length())
    {
        if (isalpha(data[i]))
        {
        data[i]=data[i];
        i++;
        }
        else
            {
            data.erase(i,1);}
    }
    cout<<data;
}