Surprise because of FullSync

Today out of a sudden I got an email question on FullSync. I didn’t recall I wrote an app called FullSync, so my first thought is, the email was sent to a wrong email address.

But something didn’t sound reasonable. If the email was sent to a wrong email address, why it was to my email address but not others?

So I Google and dig up what happened. Ended up I found that one of my commit to my forked FullSync source code was merged to the project branch by the developer, and my name is added to the special thanks of FullSync.

Well that’s something that I didn’t expect, but good that I did some contribution. However too bad that the project seems no longer active these few years.

FullSync_Code_Browse_Commits_-_2015-12-27_14.47.25

My commit at my forked source code was merged to FullSync

Advertisements

Reading UTF File with BOM to UTF-8 Encoded std::string in C++11 on Windows

I got a task from boss and during the task I need to write a C++ function to read a text file into UTF-8 encoded string. I can’t believe this easy task in high level language like VB.NET cost me 2 days to figure the solution out. Some solutions use libraries like UTF8-CPP or ICU, while some use Windows API. These ways works but I don’t quite like them because:

  1. Usability, some solutions work good for most characters but failed to handle non-BMP characters
  2. Portability, we are foreseeing our code will be migrated to Linux, use of Windows API means adding work to our migration work in the future
  3. Dependency, I don’t want to depend on third-party library, especially the C++ function later will integrated with the program which only depend on very few third-party library.

Surprisingly there is not much solution that I am satisfied from the Internet, but good that at the end I still able to figure the code out. It still used one Windows API function though, but it is much easier to port to Linux already than ten functions. Anyway here it is:

// Reading ASCII, UTF-8, UTF-16LE, UTF-16BE with auto BOM detection using C++11 on Windows platform
// Code tested on Microsoft Visual Studio 2013 on Windows 7
// Part of the code is referencing http://cfc.kizzx2.com/index.php/reading-a-unicode-utf16-file-in-windows-c/

#include <stdio.h>
#include <tchar.h>
#include <string>
#include <fstream>
#include <sstream>
#include <locale>
#include <codecvt>
#include <iostream>
#include <io.h>
#include <fcntl.h>

#define TEXT_FILE_PATH      "D:\\test.txt"
#define ENCODING_ASCII      0
#define ENCODING_UTF8       1
#define ENCODING_UTF16LE    2
#define ENCODING_UTF16BE    3

std::string readFile(std::string path)
{
	std::string result;
	std::ifstream ifs(path.c_str(), std::ios::binary);
	std::stringstream ss;
	int encoding = ENCODING_ASCII;

	if (!ifs.is_open()) {
		// Unable to read file
		result.clear();
		return result;
	}
	else if (ifs.eof()) {
		result.clear();
	}
	else {
		int ch1 = ifs.get();
		int ch2 = ifs.get();
		if (ch1 == 0xff && ch2 == 0xfe) {
			// The file contains UTF-16LE BOM
			encoding = ENCODING_UTF16LE;
		}
		else if (ch1 == 0xfe && ch2 == 0xff) {
			// The file contains UTF-16BE BOM
			encoding = ENCODING_UTF16BE;
		}
		else {
			int ch3 = ifs.get();
			if (ch1 == 0xef && ch2 == 0xbb && ch3 == 0xbf) {
				// The file contains UTF-8 BOM
				encoding = ENCODING_UTF8;
			}
			else {
				// The file does not have BOM
				encoding = ENCODING_ASCII;
				ifs.seekg(0);
			}
		}
	}
	ss << ifs.rdbuf() << '';
	if (encoding == ENCODING_UTF16LE) {
		std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> utfconv;
		result = utfconv.to_bytes(std::wstring((wchar_t *)ss.str().c_str()));
	}
	else if (encoding == ENCODING_UTF16BE) {
		std::string src = ss.str();
		std::string dst = src;
		// Using Windows API
		_swab(&src[0u], &dst[0u], src.size() + 1);
		std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> utfconv;
		result = utfconv.to_bytes(std::wstring((wchar_t *)dst.c_str()));
	}
	else if (encoding == ENCODING_UTF8) {
		result = ss.str();
	}
	else {
		result = ss.str();
	}
	return result;
}

You can also find the above code at https://gist.github.com/VeryCrazyDog/c20b2cb83896e9975d22

Building Linux RPM Package from Source Code

Sometimes for some reasons you might not be able to find RPM package that you needed on the Internet. In that case you have to build the binary by your own. Certainly an easy way is just build the binary and install it, but a better way is to build a RPM package so that it is distributable and can be under Linux package management. So below is the steps for building RPM package from source code, referencing from https://gist.github.com/fernandoaleman/1376973 with some changes.

# Step: 1
# Install rpmbuild

yum install rpm-build

# Step: 2
# Create rpmbuild folders in your home directory

mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}

# Step: 3
# Download the source file

wget -P ~/rpmbuild/SOURCES "http://git-core.googlecode.com/files/git-1.7.7.3.tar.gz"
tar -xzvf ~/rpmbuild/SOURCES/git-1.7.7.3.tar.gz -C /tmp

# Step: 4
# Prepare the .spec file, and made any necessary changes
#
# If the source file did not include .spec file, you have to prepare one by your own
ls -l /tmp/git-1.7.7.3/*.spec
cp -p /tmp/git-1.7.7.3/git.spec ~/rpmbuild/SPECS
vim ~/rpmbuild/SPECS/git.spec

# Step: 5
# Build rpm
#
# If you get any errors during build, it is usually because of dependencies. Simply
# install the dependencies with `yum install [dependency]` and run rpmbuild again.

rpmbuild -ba --define "_topdir %(echo $HOME)/rpmbuild" ~/rpmbuild/SPECS/git.spec

You can also find the above code at https://gist.github.com/VeryCrazyDog/1d1fef411d9e20f4c12b

Happy New Year 2015!

Well another year passed. Too bad I haven’t released any new version of Android app in last year and hopefully will release a new version this year. Hope you had a good time in year 2014 and happy near year 2015!

Another annual report from WordPress below.


The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 13,000 times in 2014. If it were a concert at Sydney Opera House, it would take about 5 sold-out performances for that many people to see it.

Click here to see the complete report.

Building Ncat Portable for Windows

Background

I used to use telnet for testing if the firewall working properly in the office. Telnet is good for simple test, but when comes to testing a large number of destination IP and ports it will be very time consuming. After searching on the Internet I found Netcat, a tool for testing connection. It is build-in in most Linux OS but not on Windows. So I found Ncat here as part of the Nmap for Windows by nmap.org, but too bad that it won’t execute on some Windows as it requires Microsoft Visual C++ Redistributable Package installed.

A Ncat portable is an alternative, but the website only provided a beta version of Ncat 5.59BETA1, and require you to compile your own if you would like a newer version.

They did provide a documentation on how to compile Ncat with static linking library, but if you follow the steps you will encounter a few problems which you have to troubleshoot yourselves.

So here is my steps on compiling Ncat portable after I had gone through all the trouble.If you are someone who can edit the documentation on secwiki.org, please do that for me as I don’t want to get an extra account for just editing the document :)

If you want to avoid the trouble to compile, you can download the Ncat which I compiled here, however use it at your own risk as I might have inject some malware inside :)

Building Ncat Portable for Windows

Environment

The following steps has been tested using the following environment:

Steps

  1. First, follow all steps on the original documentation at https://secwiki.org/w/Nmap/Ncat_Portable till you copied the resulted static libs and include files from “C:\OpenSSL” to the “mswin32\OpenSSL” directory in the Nmap source tree in step 5. Below is a screenshot of the original documentation in case it is edited at the time you read this article.
    Building Ncat Portable
  2. Assuming that everything went fine till now, you’re about 7 steps away from building Ncat portable.
    1. Open Nmap solution in Visual Studio from mswin32\nmap.sln and switch the build configuration to “Ncat Static” like so:
      1. Right click on Solution “nmap” in the Solution Explorer sidebar and choose “Configuration Manager“.
      2. Switch the active solution configuration to “Ncat Static“. Check the “Build” check box for project “liblua“. Make sure that the nsock, nbase and ncat projects have switched to the “Static” configuration also. Then close the “Configuration Manager”.
      3. Right click on the ncat project and select “Set as StartUp Project“.
    2. Right click on the “nsock” project in Visual Studio and click “Properties“. In “Configuration Properties” > “General” > “C/C++” > “General“, in “Additional Include Directories“, add path “..\mswin32\OpenSSL\include
    3. Right click on the “ncat” project in Visual Studio and click “Properties“. In “Configuration Properties” > “General” > “C/C++” > “General“, in “Additional Include Directories“, add path “..\mswin32\OpenSSL\include” and “..\liblua
    4. Right click on the “ncat” project in Visual Studio and click “Properties“. In “Configuration Properties” > “General” > “Linker” > “General“, in “Additional Library Directories“, add path “..\mswin32\OpenSSL\lib
    5. Expand the “ncat” project, double click the file “ncat_ssl.c” and comment out the line “#include <openssl/applink.c>
    6. Right click on the “liblua” project in Visual Studio and click “Properties“. In “Configuration Properties” > “General” > “C/C++” > “Code Generation“, set “Runtime Library” to “Multi-threaded DLL (/MD)
    7. Right click on the “ncat” project in Visual Studio and click “Build“. Alternatively you can press the F7 key to start building.

Setting Keyboard Layout to US in Chinese Traditional (Hong Kong) Language for Windows 8

I used to have US keyboard layout with my Chinese language but since Windows 8, it is just not possible. When you click the Option, you can only see all Chinese input method, but none with US keyboard layout. It is frustrating when to manage two languages with each containing one input method. It wasn’t a matter to me as I am not a Windows 8 user until I have to work on Windows Server 2012 and face this problem, especially on a server without GUI. I found my way to configure the language setting using PowerShell, and along with that I also found a way to include US keyboard layout in Chinese Traditional (Hong Kong) language as well. End up the solution to have the above screenshot result is ironically by PowerShell script, not by GUI. Anyway below is the PowerShell script that I used. You can also access the code at
https://gist.github.com/VeryCrazyDog/4cafad4183b5a684a4b9

Edit 2015-08-10: For some reason the below code is altered by WordPress during display, resulting the code not workable. If that is the case for you, try access the code using the link above.

# Won't work on Windows 7
# WARNING: This will clear your language list setting
$langList = New-WinUserLanguageList zh-Hant-HK
$langList[0].InputMethodTips.Clear()
# Keyboard layout: US
$langList[0].InputMethodTips.Add('0C04:00000409')
Set-WinUserLanguageList $langList