How to format C and C++ code in Sublime Text

How to format C and C++ code in Sublime Text

I am writing some C code on my mac using Sublime Text and I needed a quick way to format the code for better readability.

I found this package for Sublime Text named SublimeAStyleFormatter:

https://packagecontrol.io/packages/SublimeAStyleFormatter

I’ve installed it with git by going on the packages folder and cloning the git repo:

cd ~/Library/Application\ Support/Sublime\ Text\ 3/Packages/
git clone git://github.com/timonwong/SublimeAStyleFormatter.git


And that was it! I can format my C code by pressing Ctrl + Alt + F.

Cross-compiling a C application using the Android NDK

1 download ndk
export NDK=~/Downloads/android-ndk-r10e

2 Set cross-compiler for x86 (emulator)
mac:
export NDK_TOOLCHAIN=${NDK}/toolchains/x86-4.8/prebuilt/darwin-x86_64/bin/i686-linux-android-
export NDK_SYSROOT=${NDK}/platforms/android-19/arch-x86

Set cros-compiler for arm (device)
export NDK_TOOLCHAIN=${NDK}/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-
export NDK_SYSROOT=${NDK}/platforms/android-19/arch-arm

3 compile
// hello-world.c
#include <stdio.h>
int main(void)
{
printf("Hello world cross compiled on Android!\n");
 return 0;
}

make CC=${NDK_TOOLCHAIN}gcc CFLAGS=--sysroot=${NDK_SYSROOT} hello-world

4 push to test
$ adb push hello-world /data/

$ adb shell

# /data/hello-world

Hello world cross compiled on Android!

reference:
https://yaapb.wordpress.com/2012/09/27/cross-compiling-a-c-application-using-the-android-ndk/

Linux ELF 文件操作的一些工具

Linux ELF 文件操作的一些工具

1 file yourlib.so 识别文件格式

2  nm -g yourlib.so 列举目标文件中的符号,
默认输出这个文件中声明的任何函数和全局变量的名称
nm -g yourlib.so   列出所有 extern & exported symbols
nm -gC  yourlib.so    #for C++
nm -t d -l -S -v ./a.out
nm -Ca lib.so     显示所有symbols and function names
nm -D lib.so      # list symbols in the dynamic symbol table, which you can find its address by dlsym.

3 ldd yourlib.so 解析动态链接二进制文件所依赖的库文件
在OS X上，使用 otool -L yourlib.dylib 可实现类似的功能

4  objdump
Linux 自带, Mac OS 上类似为otool.
objdump -D   lib.so    #display assembler contents for all sections
objdump -tT /lib/i386-linux-gnu/libc.so.6 | grep fopen
objdump -Dslx  lib.so
objdump -x lib.so
objdump -TC lib.so

5 readelf
readelf -Ws libc.so 显示所有symbol tables
readelf -s lib.so 显示所有symbol tables
readelf -h  lib.so     # elf header
readelf -S lib.so      #sections

6 strings yourfile 用于提取文件中的字符串
strings —readix=x  yourfile     显示字符串在文件中位置
-a    扫描整个文件  
-e   也扫描Unicode字符

7   size
size /usr/sbin/httpd
Sample outputs:
   text       data        bss        dec        hex    filename
 314213      12376      13304     339893      52fb5    /usr/sbin/httpd
Where,
text - Actual machine instructions that your CPU going to execute. Linux allows to share this data.
data - All initialized variables (declarations) declared in a program (e.g., float salary=123.45;).
bss - The BSS consists of uninitialized data such as arrays that you have not set any values to or null pointers.

8 elfedit
Linux 自带， 可修改machine type, file type, osabi信息

9 hexdump -C  /bin/ls | more
linux command, 查看二进制信息

refer to:
http://www.tenouk.com/Module000linuxnm3.html

T-OTP for Google Authenticator

Google Authenticator采用的算法是T-OTP（Time-Based One-Time Password），需要知道以下三点信息：
   ＊ Key: 共享密钥
   ＊ Current Time: 当前时间输入
   ＊ HMAC-SHA1函数

共享密钥
共享密码用于在手机端上建立账户。密码内容可以是通过手机拍照二维码或者手工输入，并会被进行base32加密。

手工密码的输入格式如下：

    xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx

包含该令牌的二维码的内容是一个URL：

    otpauth://totp/Google%3Ayourname@gmail.com?secret=xxxx&issuer=Google

时间输入(当前时间)

输入的时间值来自于手机，一旦注册获得密钥后，就无需与服务器再进行通信，所以要确保手机上的时间准确，因为往后的步骤服务器可能会验证多个收到的OTP token，但时间值服务器只会取当前时间： 服务器会比对所有提交的token以确认是否有正确输入。

Hash函数

验证所用的方法是HMAC-SHA1，以一个密钥和一个消息为输入，生成一个20字节消息摘要作为输出。其算法可以简单表示为：

    hmac = SHA1(secret + SHA1(secret + input))

T-OTP与H-OTP的区别是T-OTP以当前时间作为输入，而H-OTP以自增counter(based on a seed)作为输入，该计数器使用时需要两边同步。

算法

首先，要进行密钥的base32加密。虽然谷歌上的密钥格式是带空格的，不过base32拒绝空格输入，并只允许大写。所以要作如下处理：

    original_secret = xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx 
    secret = BASE32_DECODE(TO_UPPERCASE(REMOVE_SPACES(original_secret)))

第二步要获取当前时间值，这里使用的是UNIX time函数，或者可以用纪元秒。

    input = CURRENT_UNIX_TIME()

在Google Authenticator中，input值拥有一个有效期。因为如果直接根据时间进行计算，结果将时刻发生改变，那么将很难进行复用。Google Authenticator默认使用30秒作为有效期(时间片)，最后input的取值为从Unix epoch（1970年1月1日 00:00:00）来经历的30秒的个数。

    input = CURRENT_UNIX_TIME() / 30

最后一步是进行HMAC-SHA1运算

    1.original_secret = xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx 

    2.secret = BASE32_DECODE(TO_UPPERCASE(REMOVE_SPACES(original_secret))) 

    3.input = CURRENT_UNIX_TIME() / 30 

    4.hmac = SHA1(secret + SHA1(secret + input))

HMAC运算后的结果会是20字节即40位16进制数，我们需要的是常规6位数字密码. 首先要对20字节的SHA1进行瘦身。我们把SHA1的最后4个比特数（每个数的取值是0~15）用来做索引号，然后使用从索引号开始的4个字节计算OTP。因此，索引号的操作范围是15+4=19，加上是以零开始，所以能完整表示20字节的信息。4字节的获取方法是：

    1.然后将它转化为标准的32bit无符号整数(4 bytes = 32 bit)：

    2.large_integer = INT(four_bytes)

最后再进行7位数(1百万)取整，就可得到6位数字了：

    1.large_integer = INT(four_bytes) 

    2.small_integer = large_integer % 1,000,000

这也是我们最后要的目标结果，整个过程总结如下：

    1.original_secret = xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx 

    2.secret = BASE32_DECODE(TO_UPPERCASE(REMOVE_SPACES(original_secret))) 

    3.input = CURRENT_UNIX_TIME() / 30 

    4.hmac = SHA1(secret + SHA1(secret + input)) 

    5.four_bytes = hmac[LAST_BYTE(hmac):LAST_BYTE(hmac) + 4] 

    6.large_integer = INT(four_bytes) 

    7.small_integer = large_integer % 1,000,000

一个完整可执行的GO语言程序，可以这里进行查看:
http://garbagecollected.org/2014/09/14/how-google-authenticator-works/

Linux 32-bit和64-bit编译

Environment:
Ubuntu 14.04 64-bit desktop, as guest virtual machine in virtual box
Host: MacBook Pro

在64位机器上缺省编译或运行时查找的是64位程序，但如果编译32位程序通常会遇到一些问题，这里做一个说明。

Build  32-bit C/C++ program on 64-bit Linux


sudo apt-get install g++    #to build C++ source files
sudo apt-get install g++-multilib   #to build 32-bit executable on 64-bit Linux
g++ -m32 -g -o a.out file1.cpp file2.cpp   #specify 32-bit binary

Run 32-bit executable on 64-bit Linux
要运行32位legacy程序，则需要安装相应的32位库文件(ia32-libs)。
首先添加 i386 architecture，然后安装必要的32位库。
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
sudo ./file-name
以下步骤可能也需要:
sudo apt-get update
sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0

特别的，比如需要libgc library，则需指定i386类型下载:
sudo apt-get install libgc1c2            (for 32-bit linux)
sudo apt-get install libgc1c2:i386   (for 64-bit linux)

可以用命令 “file file_name”查看执行文件类型：
file-name: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped

Shrink Virtualbox VM VDI File Size

Virtualbox  的VM image size (.vdi file)有时候会变得很大， 如何缩减文件大小呢？ 

这里我的host = MacBook Pro, guest = Win7. 虚拟机创建的时候设置的是 “Dynamically Expanding Storage” , 但以后随着增删文件操作而磁盘只会增加不会减少。

1. Run defrag in the guest (Windows only)
2. Nullify free space:
   With a Linux Guest run this:

   sudo dd if=/dev/zero of=/bigemptyfile bs=4096k
   sudo rm -rf /bigemptyfile
   

   With a Windows Guest, download SDelete from Sysinternals and run this:

   sdelete –z c:
   

3. Shutdown the guest VM
4. Now run VBoxManage's modifyhd command with the --compact option:
   With a Linux Host run this:

   vboxmanage modifyhd /path/to/thedisk.vdi --compact
   

   With a Windows Host run this:

   VBoxManage.exe modifyhd c:\path\to\thedisk.vdi --compact
   

   With a Mac Host run this:

   VBoxManage modifyhd /path/to/thedisk.vdi --compact
   

This reduces the vdi size.

一些用来加固应用的.so库

libsecexe.so 梆梆加固

libAPKProtect.so APKProtect加固,

libprotectClass.so 360加固,

libNSaferOnly.so 通付盾加固,

libnqshield.so 网秦加固,

libshell.so 腾讯加固,

ijiami.dat 爱加密加固,

libddog.so 娜迦加固,

libmobisec.so 阿里加固,

libbaiduprotect.so 百度加固

Virtualization on Android OS

Virtualisation on Android OS:

1. OKL4 Microvisor from Open Kernel Labs.  VMM runs on highest privilege, virtualising: processor(instructions,registers), memory, I/O. The other applications run on light-weight lower level.
2. VLX from VirtualLogix.
3. Mobile visor platform (MVP) from VMware, offering a hypervisor on smart mobile phones

Linux Container.

major difference between hypervisor and V-OS:
  hypervisor lies between OS and hardware, while V-OS lies above OS level. so the security of V-OS actually heavily depends on protection/obfuscation on application code.


major difference btw. security method of V-OS and app-wrapping:


Problems: the purpose of security is not very clear. what does V-OS want to protect?
algorithm/implementation or sensitive data. code obfuscation can only improve the difficulty of reverse engineering (sometimes it is sufficient). if it is sensitive data, no absolute secure solution without hardware isolation support. the best effort may be to provide multiple layer securities.

轻量级C Compiler比较

要求，纯C写就，轻量级。

TCC, LCC, PCC
结论： lcc比较适合用来写新的architecture, 即retargetable compiler. lcc规模较小，各个模块边界清晰，而且宏少，使用经验多，还有教程。 依赖少， 较易移植。缺点是不支持ARM处理器。

LCC - little C compiler
arch: x86, mips, alpha, sparc, microsoft cil
implement: ANSI C
license: LCC license
written in C
last release: sept. 2002
Retargeting能力比较强。

TCC - tiny C compiler, 
arch: X86 + ARM processor,
license: LGPL licence
implement all ANSI C(C89/C90), much of C99, many GNU C extensions including inline assembly
written in C and assembly
last release: feb. 2013
tcc有自己的汇编器与连接器

PCC: portable c compiler
arch: x86, x64
implement: C99
license: BSD
written in: C


retarget 一般放在backend做 code generation产生 machine code.

TCC的retarget支持不强。
比如一个例子： 将tinycc编译retarget to a virtual processor I designed.
则必须要实现一个code generator backend for this architecture, 类似于目录下的
i386-gen.c, arm-gen.c, and c67-gen.c 文件实现。

注意：there are apparently some issues with byte ordering, such
that the compile and target hosts must have the same endianness if you
want the output to be sensible. But I don't know if that's due to the
the *-gen implementation itself or part of the core compiler code.

另外针对 assembler, tcc does not normally generate any intermediate assembly; it goes
directly from C code to object code.


LCC
lcc code generator is based on the lburg. Some retarget examples:

retargetable C compiler for network processor
http://www.iro.umontreal.ca/~aboulham/pdfs_sources/SCI02.pdf

A PDP-11 target for lcc
http://telegraphics.com.au/sw/info/lcc-pdp11.html

Retargeting lcc for Magic-1
http://www.homebrewcpu.com/retargeting_lcc.htm

vcode or tick cc
vcode: a retargetable, extensible, very fast dynamic code generation system,
This paper appeared in PLDI '96. Slides are available here .
vcode is a portable system to generate executable code at runtime. It generates code in approximately 10 instructions per generated instruction, and is easily extendible by clients. A tutorial describing it can be obtained by clicking here. A beta version of the system is also available. To get on the vcode mailing list, please email engler@lcs.mit.edu. Click here for a bit more information. 

`C and tcc: A Language and Compiler for Dynamic Code Generation,


Quake 3

id Software's id Tech 3 engine relies on a modified version of LCC to compile the source code of each game module or third-party mod into bytecode targeting its virtual machine.[6] This means that modules are oblivious to the system beyond the system calls and limited file system scope offered by the engine, which is intended to reduce the threat posed by malicious mod authors. Another consideration is that games and mods written for the engine are portable without recompilation; only the virtual machine needs to be ported to new platforms in order to execute the modules.

id Tech 3 uses a virtual machine to control object behavior on the server, effects and prediction on the client and the user interface. This presents many advantages as mod authors do not need to worry about crashing the entire game with bad code, clients could show more advanced effects and game menus than was possible in Quake II and the user interface for mods was entirely customizable.

Virtual machine files are developed in ANSI C, using LCC to compile them to a 32-bit RISC pseudo-assembly format. A tool called q3asm then converts them to QVM files, which are multi-segmented files consisting of static data and instructions based on a reduced set of the input opcodes. Unless operations which require a specific endianness are used, a QVM file will run the same on any platform supported by Quake 3.

The virtual machine also contained bytecode compilers for the x86 and PowerPC architectures, executing QVM instructions via an interpreter.

https://github.com/id-Software/Quake-III-Arena/tree/master/q3asm
http://fabiensanglard.net/quake3/qvm.php
http://gamedev.stackexchange.com/questions/15107/how-can-i-edit-qvm-quake-virtual-machine-files
http://www.icculus.org/~phaethon/q3mc/q3vm_specs.html

Object-C Code Obfuscators

iOS application protection


Object-C是有反射（reflect）机制的，可以被用来做混淆，在一定程度上提高代码的安全性。
关键代码把核心逻辑尽量放在c层, 这样还有一个好处是iOS和android可以公用.

数据加密，服务器解析
一般对各个参数进行签名验证，对核心参数进行加密，而且防止别人进行重放攻击，一般都会将时间戳做为一个参数

Protection Techniques:

1. Control flow obfuscation e.g. ARM instruction flows are mangled with redundant instructions to try to hide the original purpose of the code,
2. Class and Method renaming - renames your methods and classes to meaningless names although you have to be careful where this is used as you can easily break your app because the Objective-C runtime is expecting to find certain names,
3. String encryption - all static strings in the app are encrypted and code is inserted to decrypt the strings just before use in order to make static analysis harder
4. Anti-debug - code is inserted to break the usual debuggers (not always successfully),
5. Anti-tamper - usually builds a network of checksums that protect the binary code from modification,
6. Objective-C runtime protection - usually checks obj-c registered method implementations to make sure that they are in the app and haven't been 'swizzled'. 
7. encrypt constant, strings, decrypt in real-time
8. encrypt local stored files

Detection Techniques:

• detect debug status
• detect jail-break status
• integrity check of data or piece of code

Object-C code obfuscators:

• EnsureIT for Apple iOS
• Contaxiom Code Protection
• Arxan,
• Metaforic,
• Cryptanium
• iOS class-guard to rename class /method names

RE tools:

• class-dump
• gdb
• IDA pro/hopper disassembler

关于ARM的built-in security

iOS和Android smartphone 的ARM芯片都内置实现了TrustZone技术，提供TEE安全运行环境（包含TrustedOS和硬件支持）， 与Android OS隔离，因此理论上可以彻底阻止软件层面上来的攻击。 这里TrustedOS可以看作一个Hypervisor，监视上层的”TrustedAPP”.

因为TrustedOS属于定制build和安装，它是不能从app store上下载的，所以应用提供者还必须将智能手机定制为TrustedOS firmware才可以使用. 看起来移植性不太好哈，不过还是可以做一个支持通用接口的TEE kernel用来安装其他认证应用。Trustonic和三星就在做。

当然ARM也不落后，针对64-bit ARMv8 系列芯片，它给出了一个开源的项目(BSD license) the source code is available on Github.

其他开源的TrustOS:
OVOS : http://www.openvirtualization.org/  也是隔离了secure world and normal world.


reference:
http://www.androidauthority.com/arms-built-security-might-just-get-rid-password-397924/

为反编译Android应用设置Linux环境

Linux environment setup

ubuntu 14.04 64-bit virtual machine on virtual box
1. shared folder:   in virtual box setting, create two shared folder “Documents”, “Downloads” on mac os, set full r/w rights and permanent. then in unbuntu’s mnt directory you will see “sf_Downloads” and “sf_Documents” ( with AdditionIn package installed ).
2. however, you may need to run “sudo adduser fanghui vboxsf” to add yourself to vboxsf group, and restart ubuntu guest.


android apktool
http://ibotpeaches.github.io/Apktool/install/
1. Check java 1.7 installed or not
       $java -version
       $sudo apt-get install default-jre  (or openjdk-7-jre)

2. Download Linux wrapper script (https://raw.githubusercontent.com/iBotPeaches/Apktool/master/scripts/linux/apktool,  Right click, Save Link As apktool)

3. Download apktool-2 (https://bitbucket.org/iBotPeaches/apktool/downloads, find newest here)

4. Make sure you have the 32bit libraries (ia32-libs) downloaded and installed by your linux package manager, if you are on a 64bit unix system.
(This helps provide support for the 32bit native binary aapt, which is required by apktool)

5. Rename downloaded jar to apktool.jar

6. Move both files (apktool.jar & apktool) to /usr/local/bin (root needed)
Make sure both files are executable (chmod +x)

7. Try running apktool via cli
   example:   $apktool d flappy-bird.apk
  will unzip the file structure
  find “Fake Flappy birds on Android” on http://androidmalwaredump.blogspot.sg/


dex2jar
   download dex2jar from http://sourceforge.net/projects/dex2jar/files/
   extract to a folder
        $unzip -x dex2jar-2.0.zip  -d /home/your_folder
   convert apk to jar file  ( you might need chmod +x d2j* )
        $sh /home/your_folder/dex2jar-2.0/d2j-dex2jar.sh  flappy_bird.apk


jd-gui
    download from jd.benow.ca
  it is a jar file “jd-gui-1.1.0.jar", so run with
     $java -jar jd-gui-1.1.0.jar
 a gui will pop up.

jad
jad is a decompiler tool similar to jd-gui, except command line.
download from http://varaneckas.com/jad/
somehow i cannot run jad 1.5.8e for linux on intel, so i choose jad 1.5.8e for linux (statically linked) instead.


boomerang (not mature)
download boomerang-linux-alpha-0.3 from http://boomerang.sourceforge.net/download.php
it requires libgc. when I try to run the program, I get this error:
./boomerang: error while loading shared libraries: libgc.so.1: cannot open shared object file: No such file or directory

You need to install libgc, something like

sudo apt-get install libgc1c2
sudo apt-get install libgc1c2:i386   (for 64-bit linux)

boomerang also requires libexpat1
sudo apt-get install libexpat1:i386

ok.

====
Note: run 32-bit executable on 64-bit linux

use “file file_name” to view exe type.

file-name: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped

firstly install ia32-libs.
sudo apt-get install ia32-libs
or
sudo apt-get update
sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0
or
To run 32bit executable file in a 64 bit multi-arch Ubuntu system, you have to add i386 architecture and also you have to install libc6:i386,libncurses5:i386,libstdc++6:i386 these three library packages.
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
sudo ./file-name