Unicode supported isdigit and isspace function

I have the following code.

// mfc.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include "mfc.h"

#ifdef _DEBUG
#define new DEBUG_NEW
#endif

#include <cctype>
#include <string>
#include <sstream>
#include <tchar.h>
#include <iostream>
#include <Strsafe.h>
#include <algorithm>
#include <cmath>
#include <limits>
#include <functional>
#include <cassert>

std::wstring toSt开发者_StackOverflowringWithoutNumerical(const std::wstring& str) {
    std::wstring result;

    bool alreadyAppendSpace = false;
    for (int i = 0, length = str.length(); i < length; i++) {
        const TCHAR c = str.at(i);
        if (isdigit(c)) {
            continue;
        }
        if (isspace(c)) {
            if (false == alreadyAppendSpace) {
                result.append(1, c);
                alreadyAppendSpace = true;
            }
            continue;
        }
        result.append(1, c);
        alreadyAppendSpace = false;
    }

    return result;
}


// The one and only application object

CWinApp theApp;

using namespace std;

int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
    int nRetCode = 0;

    // initialize MFC and print and error on failure
    if (!AfxWinInit(::GetModuleHandle(NULL), NULL, ::GetCommandLine(), 0))
    {
        // TODO: change error code to suit your needs
        _tprintf(_T("Fatal Error: MFC initialization failed\n"));
        nRetCode = 1;
    }
    else
    {
        // TODO: code your application's behavior here.
    }

    std::wstring me = toStringWithoutNumerical(_T("My Leg 1 Long"));
    AfxMessageBox(me.c_str());

    // Crash!
    std::wstring he = toStringWithoutNumerical(L"我的脚1盘");
    AfxMessageBox(he.c_str());

    return nRetCode;
}

For 1st message box,

My Leg Long

will be shown.

For 2nd message box, crash will happen, with assertion fail at isctype.c

_ASSERTE((unsigned)(c + 1) <= 256);

How I can get a standard function (isdigit, isspace...), to support unicode out of 256 range?

iswdigit?

http://www.opengroup.org/onlinepubs/007908775/xsh/iswdigit.html
http://msdn.microsoft.com/en-us/library/fcc4ksh8(VS.71).aspx

also, look up iswspace ;)

There are also a C++ template, locale-aware versions of isspace and isdigit, located in <locale> header if you want to go multiplatform.

The correct way to handle Unicode characters is extremely complex. However, if you just want to check a few character categories, you can generally make do without too much trouble.

The Unicode interface in POSIX (Unix) is u_isXYZ such as u_isspace().

http://icu-project.org/apiref/icu4c/index.html

#include <unicode/uchar.h>

...
if(u_isspace(c))
{
   ... // this is a space character
}
...

From what I can see the ICU library is available under MS-Windows so you should be able to use it too.

Remember, though, that wchar_t under MS-Windows is only 16 bits, so you have to handle surrogates yourselves (values between 0xD800 and 0xDFFF represent characters between 0x10000 and 0x10FFFF--although those are not used as much if you do not deal with Asian languages).

The default iswspace() is locale bound and will definitively not return the same results as the proper Unicode functions.