Compile from source and install it in your home.

Since several months, I would always like to build software from source. It began when I would install vim 8.0 but no packages for Ubuntu were available. I wrote an article about it.

I began by building from source and install it with sudo. But, I had some issues when the package was updated because there were some conflicts.
For these reasons, I decided to build from source and to install the binaries in a separated directory in my home, called $HOME/.bin/. When the binaries are installed in this directory, I only need to extend my PATH with the appropriate subdirectories.

Here some example.

Git

In Ubuntu, the version provided by packages is 2.7.4 from 2016. The current version if 2.13. All source codes I need are in a directory called $HOME/.source. I use the following commands to build git from source and install it locally.

mkdir -p $HOME/.source
git clone https://github.com/git/git $HOME/.source/git
cd $HOME/.source/git
./configure --prefix=$HOME/.bin/git
make -j 8
make install

The usual commands to build C programs are ./configure && make && make install and you can pass some arguments like –prefix=path to ./configure to change the directory where the binaries must be installed.

Emacs

Even if I’m a vim user since several years, I switched to Emacs some months ago. I also want to have the latest version and I use the following commands.

mkdir -p $HOME/.source
git clone https://github.com/emacs-mirror/emacs $HOME/.source/emacs
cd $HOME/.source/emacs
./configure --prefix=$HOME/.bin/emacs
make -j 8
make install

Usually you will easily find the commands to build. If configure is used, you can often use the flag –prefix (it is recommended by the GNU project to always add it when you provide a configure script). I personally recommend it to avoid conflicts and to easily remove the version you don’t want to use anymore.

Hope it can help you.

OCaml/Reason Compiler Hacking sessions. First announcement…

It’s time for me to organize my first event about OCaml (and in the same time, Reason), and more specifically something like the Compiler Hacking Sessions OCamlLabs holds in Cambridge.

It would be the first week-end of July 2017 (1st and 2nd) in my apartment in the south of Belgium near Ath (the address will be given to people who will participate, I don’t want to publish it on the Internet). Everyone can come, even if you don’t have any knowledge about compilers or type systems.
This week-end won’t be only focused on OCaml Compilers Hacking. I mainly organize this event to meet other developers, talk about technologies, things we like, share cultures, taste beers, eat Belgian fries (not French fries, it’s not the same, and fries are Belgian, not French…). But of course, code and learn will be the most important.

More information will come later like links to begin to hack with type systems, information about compilers. I will also post some hacks which can be interesting in the OCaml/Reason community. To OCaml/Reason maintainers, contact me if you have suggestions (maybe add some labels on the GitHub repository for contributions). Contributions can also be about the ocaml.org website, documentation, write documentations for next sessions, etc. No idea for the moment what we will work on.
I will also post later more practical information how to come to my apartment (from Brussels, it’s easy, it only takes one hour. I will help you, don’t worry about that).

The event will be free about registration and hosting. As it’s the first time I organize this type of events, I will only ask you to pay/participate for food and drinks (a special beer will be offered). Belgium is not really expensive, specially in my region. Around 20€ is enough for the weekend, we will see during the weekend how much food costs. I hope later I will be able to offer it (yes, I plan to reorganize it often). I will also ask to participants to register on an EventBrite event. People who are interested can already send me a message.

II can host maximum 6 people, and it would be the place for hacking for 8-9 people. If someone has a bigger place (with good Internet connection, no firewall, no Internet connection restrictions) and wants to host the event, please contact me. More people we can have, better it is. If you live near Ath and want to host some people for the night, you can also contact me. About the apartment, don’t worry about Internet connection (10Mbs) and places. We have two rooms around 15m2 where we can hack.

In the next few weeks, I will invest in some interesting stuffs like 50″ TV to make presentations, a huge meeting table and maybe hardwares to work on different architectures. I already have a raspberry pi 3, a rasp zero, a rasp 1B, I can have some Arduino microcontrollers, etc. Of course, you need your laptop and something to sleep on.

If you want to help to organize this event, or if you want to sponsor it by giving/lending stickers, hardware or anything else (screens can be very interesting, I have only two at home), you can contact me. I already thank OCamlLabs for stickers they will send to me.

More will come in the next few weeks. You can follow me on Twitter (@dwillems42) for up-to-date info or send me an email to contact@danny-willems.be. Hope there will be interested people!

How to configure and install vim from source?

Vim 8.0 has been released some weeks ago and maybe you want to install it. Sometimes, binaries in packages are often not up-to-date or not configured with python. Here the command to install it from source with python et python3 options (useful if you use YouCompleteMe):

git clone https://github.com/vim/vim
cd vim
./configure --enable-pythoninterp --enable-python3interp
make
sudo make install

You can also add other options. You can list them with:

./configure --help

Personally, I use the following flags

./configure --with-features=huge \
            --enable-multibyte \
            --enable-rubyinterp=yes \
            --enable-pythoninterp=yes \
            --enable-python3interp=yes \
            --enable-perlinterp=yes \
            --enable-luainterp=yes \
            --enable-gui=gtk2 \
            --enable-cscope \
            --prefix=$INSTALL_COMPILED_DIRECTORY/vim

where $INSTALL_COMPILED_DIRECTORY is another directory in my home where the binaries will be installed instead of /usr. In this case, I don’t need to use sudo and it doesn’t affect the standard installation of vim.
 

OCaml + Cordova = more secured, typed and hybrid mobile applications.

Since several months, I began to be interested in mobile development. I found a job in Brussels in July 2015 where I learned how to develop hybrid mobile applications (one code = available for multiple mobile platforms) with Cordova and web technologies. I found it very interesting and after several month I continue to develop applications with this technology. I discovered a very great community about mobile development and some awesome frameworks like Ionic.

The majority of these frameworks use JavaScript as programming language but I don’t really like this language because you have no types, some weird things (equality between string and integer), parameters are sent as undefined if not passed, etc. I don’t really like to develop applications with JavaScript because it’s very ugly (even if I think it’s OK for prototyping, but not in production).

I discovered OCaml at the university, a very powerful programming language with inferred static type, type checking at compilation time, an extraordinary community and… a compiler from OCaml to JavaScript! So, I wanted to use this language to develop mobile applications with Cordova: it will be my university project for a semester.

The goal of my project is to be able to use native components of smartphones such like accelerometer, camera, send sms, etc in OCaml.

What are Cordova, js_of_ocaml and gen_js_api?

  • Cordova allows you to develop hybrid mobile applications using web technologies such as HTML, CSS and JavaScript. For more information, see the official website. Through Cordova plugins, you can access to the native components. To learn how to make Cordova plugins, see the official tutorial. You can find the official Cordova plugin list here.
  • js_of_ocaml provides a compiler from OCaml to JavaScript. Since Cordova applications use JavaScript, js_of_ocaml provides a way to develop mobile application using OCaml. For more info, see the Ocsigen project which contains js_of_ocaml.
  • gen_js_api aims at simplifying the creation of OCaml bindings for JavaScript libraries. It must currently be used with the js_of_ocaml compiler, although other ways to run OCaml code “against” JavaScript might be supported later with the same binding definitions (for instance, Bucklescript, or direct embedding of a JS engine in a native OCaml application).

All bindings are developed with gen_js_api and aims to be functional, typed and very close to the JavaScript interface.

Continue reading

Ocsigen: how to compile an Eliom project. Understand the compilation process in deep.

It’s time to write my first article on Ocsigen and especially on Eliom. I began a week ago my internship in BeSport, a social network centralized on sports and entirely developed in OCaml using the Ocsigen projects so I needed to learn how Ocsigen works in general.

Ocsigen? What’s that? It’s an atom no?

Yes, oxygen is a chemical element but Ocsigen is also an OCaml web framework began in 2004.

Wait, another web framework? There are plenty of existing web frameworks.

Yeah, I agree. But Ocsigen is different in some cases. First, Ocsigen is entirely written in OCaml: no PHP, no JavaScript, no HTML. Second, Ocsigen contains independent «small» projects which, together, form a very elegant framework. Here some of these sub-projects:

  • Ocsigenserver: web server entirely written in OCaml. It supports HTTPS, multiple hosts (virtual hosts), proxy, content compression, access control, authentication, etc. Everything you need is implemented and very easy to configure. Ocsigenserver allows you to develop modules to add functionalities to the server.
  • Js_of_ocaml: a compiler OCaml bytecode to JavaScript. It allows you to write OCaml program and to compile it in JavaScript so you can write web application entirely in OCaml.
  • Eliom: a high level library to build client side and server side applications. In few lines of code and in the same file, you can write complex websites and the server side associated. It uses the strong OCaml type system to create route, html pages, data exchange, session mechanisms, etc. It uses new concepts in web programming which are very interesting and are up-to-date with modern needed.
  • Lwt: cooperative programming in OCaml. It’s popular in the OCaml community even in non-web project.
  • TyXML: write typed XML tree (in particular HTML).
  • and more… You can find all Ocsigen sub-projects here

OCaml on Raspberry Pi

This morning, I wanted to play with my Raspberry Pi which is, since a long time, not used. After downloading the last raspbian image and installing it with dd (see here), I wanted to make some scripts, to do some programming. I read some articles like this one from Jane Street installing OCaml on a Raspberry Pi and doing some coding stuff with Async. So, I had my morning project: install OCaml and opam on my Raspberry Pi 1 model B+ along with opam packages.

Installing OCaml

The first step is to install some dependencies like git and m4 to be able to install opam. Normally, git is pre-installed but not m4.

sudo apt-get install m4

After that, OCaml 4.01.0 has a package and can be installed with apt-get:

sudo apt-get install ocaml

The installation takes a few minutes but after that, we can use OCaml without problem.

OCaml on Raspberry Pi

OCaml on Raspberry Pi

Clone, compile and install opam

It’s time to be able to use opam to install our favorite packages. We will install opam from source to have an optimized installation. We need to configure, install external libraries, compile opam before being finally able to install it. All commands are given on the official github repository:

git clone https://github.com/ocaml/opam
cd opam
./configure
make lib-ext
make
sudo make install

Cloning and configuring are fast. The longest steps are installing external libraries and compiling opam: it takes several minutes.
If you are using a Raspberry pi 3, which has a quad core CPU, you can use

make -j 4

.
These commands will install the last version of OPAM (which is for the moment 2.0) but maybe you want OPAM 1.2. In this case, use:

git clone https://github.com/ocaml/opam
cd opam
git checkout -b 1.2 origin/1.2
./configure
make lib-ext
make
sudo make install

We also need to launch opam init, which takes also several minutes:

opam init

Install the last OCaml version

Before using any package, I would like to switch to a more recent OCaml version, such as the last release: 4.03.0. We can easily do this with opam switch.

opam switch 4.03.0

It takes a very long time because the Raspberry Pi needs to compile OCaml. CPU is always at 100% but it doesn’t take too much RAM, the entire system taking 200 MB max. You will be stuck at this step for a long time.

opam switch 4.03.0 stuck

And after around 1 hour, we have an fresh OCaml 4.03.0 installation on our Raspberry Pi:

ocaml 4.03.0

ocaml 4.03.0

Install an opam package: js_of_ocaml

As a fan of js_of_ocaml, I would like to try it on my Raspberry Pi.

opam install js_of_ocaml

Still no issues with the installation but it takes also a long time due to camlp4 in particular but everything is OK, without any memory issue.

How to set up a multi-language and collaborative wordpress blog

A multi-language and collaborative blog

When I decided to write a blog with WordPress, I had some difficulties choosing to write in French or in English and I would something collaborative.
I decided to begin in English because I thought I would have more visitors and it would be a good writing exercise.

After some days and searches about French blogs, I figured out that there were a lot of French blogs without English translations. Some argue they wanted to keep a French community and others because they don’t have knowledge to write in English.

Besides these reasons, I found some bad quality articles and I wanted to improve that.

So, it was finally decided! I’ll write in both French and English.
I know I won’t be able to write in both languages, and I’ll make a lot of errors in English.

Addicted to the open source philosophy, I found a solution which is to put my articles on a collaborative platform such as GitHub and allow pull requests from people to improve my articles. On top of that, my articles could be easily translated in other languages.

Here are two plugins I found to setup a collaborative and multi-language blog.

It’s not a tutorial to learn how to use these plugins. There’re plenty of tutorial on the web.

Multi language: Polylang

The first step is to install a plugin to allow to redact in several languages. I chose Polylang because it’s easy to install and doesn’t perform persistent changes.

Collaborative: WordPress Github Sync

GitHub is a good platform to share our code, and it’s the platform I chose.
After some searches about a plugin which allows to export and import from GitHub, I found WordPress Github Sync.
It’s also easy to install and it’s very simple to use.

Merge these two plugins

I wasn’t entirely satisfied when I began to use them. Individually, it’s perfect, but there’re some inconveniences when you try to export to GitHub the same article written in French and in English.
By default, WordPress GitHub Sync exports as Y-m-d-article_name. The problem is you have to look at the article name to know which language is. I wanted something like lang-article_name, the date isn’t important for me.

I decided to modify the GitHub plugins (which can be found here). It exports as lang-article_name by parsing the permalink (supposed to be in the form http://domain/lang/article_name).
I added a get_lang method into the post.php file and changed the github_filename method.

The result can be found on my GitHub: dannywillems, and my articles in this repository

How to set up a mobile build server: step 3

  1. Introduction
  2. Server-side and client-side: (Optional) Create and configure an user on the build server
  3. Server-side: Install and configure the Android SDK
  4. Server-side: Install and configure cordova/ionic
  5. Client-side: Automate the build process

Server-side: Install and configure cordova/ionic

This step will be very short. We would like to install globally cordova and/or ionic without root privileges. By default, we need to use sudo with npm install -g (to install globally).
We can avoid these root command by changing the directory where the npm packages are installed.

Each command is executed on the server

Change npm packages installation directory

The new directory will be in your home and called .npm-packages. Create the directory and change the npm prefix by using

mkdir -p ~/.npm-packages
npm config set prefix "~/.npm-packages"

The executable files like cordova and ionic will now be in ~/.npm-packages/bin. We need to add this directory to your PATH to be able to use directly the command.

echo "export PATH=$PATH:~/.npm-packages/bin" >> ~/.${SHELL##*/}rc

You need to reload your shell configuration. You can do it by closing and reopening your terminal or by using the command

source ~/.${SHELL##*/}rc

Before continuing, check you have the directory ~/.npm-packages/bin in your PATH with

echo $PATH

Install cordova and ionic without root privileges

Now, we can install cordova and/or ionic. Here, we will install only cordova.

npm install -g cordova

What’s next?

It’s time to write our build script.

Go to the last part (coming soon). I’m working on several projects now, I don’t have time to write this last part.

Script

You can download it here.

How to set up a mobile build server: step 2

  1. Introduction
  2. Server-side and client-side: (Optional) Create and configure an user on the build server
  3. Server-side: Install and configure the Android SDK
  4. Server-side: Install and configure cordova/ionic
  5. Client-side: Automate the build process

Server-side: Install and configure the Android SDK

It’s time now to install the Android SDK with the command line interface.

If you want to set up a build server for Mac OS X, you only need to download XCode and the iOS SDK will be downloaded. For LaTeX, see the appropriate install section for your distribution. At the end, some links will be listed for more informations about these alternatives

Each commands must be executed on the server so first connect you on the server.

Download and install the Android SDK

Go to the Android website to download the SDK. Copy the link for the Linux version and use wget.
The actual version is r24.4.1, so the command is

    wget http://dl.google.com/android/android-sdk_r24.4.1-linux.tgz -O /tmp/android-sdk.tgz
I wrote a small python script which parse the html page and get the link of the most recent SDK version. Download it

We extract the android sdk into a new hidden directory in the home named .android-sdk.

    mkdir -p ~/.android-sdk
    tar xvf /tmp/android-sdk.tgz -C ~/.android-sdk

Configure the PATH

Now we have to add the android executables such as android (SDK Manager) and adb to the PATH. An ANDROID_HOME variable is also necessary.

    echo "ANDROID_HOME=~/.android-sdk/android-sdk-linux" >> ~/.${SHELL##*/}rc
    echo "PATH=$PATH:~/.android-sdk/android-sdk-linux/tools:~/.android-sdk/android-sdk-linux/platform-tools" >> ~/.${SHELL##*/}rc

Reload your shell configuration file with

    source ~/.${SHELL##*/}rc

to update the PATH for the current ssh session.

Choose SDK API to install

SDK is installed and the PATH is configured. Now we have to install which SDK versions we want to install.

The SDK manager can be managed with the android command. By default, it launched the SDK Manager window.
To use the SDK manager in command line, you have to add the –no-ui at the end of each andoid command.

For example, to list all sdk versions, you use

    android list sdk --no-ui

and you get a similar output than

Screen Shot 2015-12-31 at 22.02.53

The numbered list in the output let you choose which SDK API you want to install with the command android update sdk -u -a -t .

First we update the sdk with the command

    android update sdk --no-ui

Accept the licence and as it’s the first time you launch this command, it will download the last Android SDK API (Now: Android 6.0 ie API 23).

If you want to install more SDK API, use

    android update sdk -u -a -t <package no.>

where is the number in the previous numbered list. (-u is a shortcut –no-ui, -t for filters and -a for all, see android -h).

With my previous screenshot, if you want to install the API for Android 5.1.1 (API 22), you use

    android update sdk -u -a -t 4

Install the API you want.

And now ?

The SDK is installed, only using the command line. Great !
Now it’s time to configure cordova and ionic. It would be a small step.

Go to the third part

Script

Download it. Don’t forget: it’s a server-side script !

How to set up a mobile build server: step 1

  1. Introduction
  2. Server-side and client-side: (Optional) Create and configure an user on the build server
  3. Server-side: Install and configure the Android SDK
  4. Server-side: Install and configure cordova/ionic
  5. Client-side: Automate the build process

Server-side and client-side: (Optional) Create and configure an user on the build server

In this part, we will create a new user and configure ssh access. It’s an optional part because you can use an existing user.
I chose to separate the user because I think an user on a server must create for only one thing.
On top of that, as I said in the introduction, this method can be used for a C, LaTeX (or anything else) build server. An user for each build method is better in this case.

The user will be called android-sdk.

It’s the only part of this tutorial you need root access to create the user

Server-side: Create the user

Connect to the server and create a new user:

adduser android-sdk

You’ll be prompted for some informations such as a name, full name, and a password.

Client-side: create a ssh key and add it to the server

To avoid having to type the password each time we push, we create a ssh key and add it to the android-sdk’s authorized keys.

ssh-keygen -t rsa -f ~/.ssh/android-sdk
ssh-copy-id -i ~/.ssh/android-sdk android-sdk@[your-server]
Replace [your-server] by your server ip/alias/domain name. If you’re on Mac OS X, install ssh-copy-id with brew (brew install ssh-copy-id).
It’s recommended to have different ssh keys for each ssh user that is for github.com, bitbucket.com or other users on your server.

Client-side: add a rule in the ssh config

If you followed the previous command, you must add a rule in the ~/.ssh/config file to match the appropriate ssh key file.
Open with your favorite text editor (vim of course :D) and add these lines

Host build-server
Hostname [your-server-adress/ip/alias]
IdentityFile ~/.ssh/android-sdk
User android-sdk
See this excellent tutorial for more informations about the ssh config file. Don’t forget to change the user if you’re using another user. Same for the ssh key file.

The host will be used when you’ll clone the repository. You will use somethink like

git clone android-sdk@build-server:~/my_amazing_project.git

Scripts

As promised, I give you scripts to automate the configuration. There’re two scripts: one for the server and one on the client machine.

Server

We only need to create the user. I only add a variable to let you choose your user name.

## Change it if you want another username.
## Don't forget to change in the client script too.
USERNAME = android-sdk

## DON'T EDIT THESE FOLLOWING LINES
adduser $USERNAME
##

Client

We need to create a ssh key, add the ssh key to the server and modify the ssh config file. If you changed the username in the script server.

##### Change it if you want another username.
## Don't forget to change in the server script too.
## [default = android-sdk]
USERNAME = android-sdk

## Your server IP adress or alias
HOSTNAME = danny-willems.be

## The host name you want to use when you'll clone
## [default = build-server]
HOST = build-server

## SSH key name
## [default = android-sdk]
SSH_KEY_NAME = android-sdk

## DON'T EDIT THESE FOLLOWING LINES
## Create SSH keys
ssh-keygen -t rsa -f ~/.ssh/$SSH_KEY_NAME

## Copy to the server
ssh-copy-id -i ~/.ssh/$SSH_KEY_NAME $USERNAME@$HOSTNAME

## Modify the ssh config file
echo 'Host $HOSTn Hostname $HOSTNAMEn IdentityFile ~/.ssh/$SSH_KEY_NAMEn User $USERNAME' >> ~/.ssh/config

And now ?

OK, user configured. Now, let’s download and install the Android SDK in our new user environment.

Go to the second part