8021b5241c
This adds changes to the rebar3 expression that patch rebar3 to force it to be hermetic. Now, by default, rebar3 literally can't download anything. A 'rebar3-open' expression was added for those folks whe want the normal rebar3.
305 lines
11 KiB
XML
305 lines
11 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook"
|
||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||
xml:id="users-guide-to-the-erlang-infrastructure">
|
||
|
||
<title>User's Guide to the Erlang Infrastructure</title>
|
||
<section xml:id="build-tools">
|
||
<title>Build Tools</title>
|
||
<para>
|
||
By default Rebar3 wants to manage it's own dependencies. In the
|
||
normal non-Nix, this is perfectly acceptable. In the Nix world it
|
||
is not. To support this we have created two versions of rebar3,
|
||
<literal>rebar3</literal> and <literal>rebar3-open</literal>. The
|
||
<literal>rebar3</literal> version has been patched to remove the
|
||
ability to download anything from it. If you are not running it a
|
||
nix-shell or a nix-build then its probably not going to work for
|
||
you. <literal>rebar3-open</literal> is the normal, un-modified
|
||
rebar3. It should work exactly as would any other version of
|
||
rebar3. Any Erlang package should rely on
|
||
<literal>rebar3</literal> and thats really what you should be
|
||
using too.
|
||
</para>
|
||
</section>
|
||
|
||
<section xml:id="how-to-install-erlang-packages">
|
||
<title>How to install Erlang packages</title>
|
||
<para>
|
||
Erlang packages are not registered in the top level simply because
|
||
they are not relevant to the vast majority of Nix users. They are
|
||
installable using the <literal>erlangPackages</literal> attribute set.
|
||
|
||
You can list the avialable packages in the
|
||
<literal>erlangPackages</literal> with the following command:
|
||
</para>
|
||
|
||
<programlisting>
|
||
$ nix-env -f "<nixpkgs>" -qaP -A erlangPackages
|
||
erlangPackages.esqlite esqlite-0.2.1
|
||
erlangPackages.goldrush goldrush-0.1.7
|
||
erlangPackages.ibrowse ibrowse-4.2.2
|
||
erlangPackages.jiffy jiffy-0.14.5
|
||
erlangPackages.lager lager-3.0.2
|
||
erlangPackages.meck meck-0.8.3
|
||
erlangPackages.rebar3-pc pc-1.1.0
|
||
</programlisting>
|
||
<para>
|
||
To install any of those packages into your profile, refer to them by
|
||
their attribute path (first column):
|
||
</para>
|
||
<programlisting>
|
||
$ nix-env -f "<nixpkgs>" -iA erlangPackages.ibrowse
|
||
</programlisting>
|
||
<para>
|
||
The attribute path of any Erlang packages corresponds to the name
|
||
of that particular package in Hex or its OTP Application/Release name.
|
||
</para>
|
||
</section>
|
||
<section xml:id="packaging-erlang-applications">
|
||
<title>Packaging Erlang Applications</title>
|
||
<section xml:id="rebar3-packages">
|
||
<title>Rebar3 Packages</title>
|
||
<para>
|
||
There is a Nix functional called
|
||
<literal>buildRebar3</literal>. We use this function to make a
|
||
derivation that understands how to build the rebar3 project. For
|
||
example, the epression we use to build the <link
|
||
xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>
|
||
project follows.
|
||
</para>
|
||
<programlisting>
|
||
{stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
|
||
|
||
buildRebar3 rec {
|
||
name = "hex2nix";
|
||
version = "0.0.1";
|
||
|
||
src = fetchFromGitHub {
|
||
owner = "ericbmerritt";
|
||
repo = "hex2nix";
|
||
rev = "${version}";
|
||
sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
|
||
};
|
||
|
||
erlangDeps = [ ibrowse jsx erlware_commons ];
|
||
}
|
||
</programlisting>
|
||
<para>
|
||
The only visible difference between this derivation and
|
||
something like <literal>stdenv.mkDerivation</literal> is that we
|
||
have added <literal>erlangDeps</literal> to the derivation. If
|
||
you add your Erlang dependencies here they will be correctly
|
||
handled by the system.
|
||
</para>
|
||
<para>
|
||
If your package needs to compile native code via Rebar's port
|
||
compilation mechenism. You should add <literal>compilePort =
|
||
true;</literal> to the derivation.
|
||
</para>
|
||
</section>
|
||
|
||
<section xml:id="hex-packages">
|
||
<title>Hex Packages</title>
|
||
<para>
|
||
Hex packages are based on Rebar packages. In fact, at the moment
|
||
we can only compile Hex packages that are buildable with
|
||
Rebar3. Packages that use Mix and other build systems are not
|
||
supported. That being said, we know a lot more about Hex and can
|
||
do more for you.
|
||
</para>
|
||
<programlisting>
|
||
{ buildHex }:
|
||
buildHex {
|
||
name = "esqlite";
|
||
version = "0.2.1";
|
||
sha256 = "1296fn1lz4lz4zqzn4dwc3flgkh0i6n4sydg501faabfbv8d3wkr";
|
||
compilePort = true;
|
||
}
|
||
</programlisting>
|
||
<para>
|
||
For Hex packages you need to provide the name, the version, and
|
||
the Sha 256 digest of the package and use
|
||
<literal>buildHex</literal> to build it. Obviously, the package
|
||
needs to have already been published to Hex.
|
||
</para>
|
||
</section>
|
||
</section>
|
||
<section xml:id="how-to-develop">
|
||
<title>How to develop</title>
|
||
<section xml:id="accessing-an-environment">
|
||
<title>Accessing an Environment</title>
|
||
<para>
|
||
Often, all you want to do is be able to access a valid
|
||
environment that contains a specific package and its
|
||
dependencies. we can do that with the <literal>env</literal>
|
||
part of a derivation. For example, lets say we want to access an
|
||
erlang repl with ibrowse loaded up. We could do the following.
|
||
</para>
|
||
<programlisting>
|
||
~/w/nixpkgs ❯❯❯ nix-shell -A erlangPackages.ibrowse.env --run "erl"
|
||
Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
|
||
|
||
Eshell V7.0 (abort with ^G)
|
||
1> m(ibrowse).
|
||
Module: ibrowse
|
||
MD5: 3b3e0137d0cbb28070146978a3392945
|
||
Compiled: January 10 2016, 23:34
|
||
Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
|
||
Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
|
||
debug_info,debug_info,nowarn_shadow_vars,
|
||
warn_unused_import,warn_unused_vars,warnings_as_errors,
|
||
{i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
|
||
Exports:
|
||
add_config/1 send_req_direct/7
|
||
all_trace_off/0 set_dest/3
|
||
code_change/3 set_max_attempts/3
|
||
get_config_value/1 set_max_pipeline_size/3
|
||
get_config_value/2 set_max_sessions/3
|
||
get_metrics/0 show_dest_status/0
|
||
get_metrics/2 show_dest_status/1
|
||
handle_call/3 show_dest_status/2
|
||
handle_cast/2 spawn_link_worker_process/1
|
||
handle_info/2 spawn_link_worker_process/2
|
||
init/1 spawn_worker_process/1
|
||
module_info/0 spawn_worker_process/2
|
||
module_info/1 start/0
|
||
rescan_config/0 start_link/0
|
||
rescan_config/1 stop/0
|
||
send_req/3 stop_worker_process/1
|
||
send_req/4 stream_close/1
|
||
send_req/5 stream_next/1
|
||
send_req/6 terminate/2
|
||
send_req_direct/4 trace_off/0
|
||
send_req_direct/5 trace_off/2
|
||
send_req_direct/6 trace_on/0
|
||
trace_on/2
|
||
ok
|
||
2>
|
||
</programlisting>
|
||
<para>
|
||
Notice the <literal>-A erlangPackages.ibrowse.env</literal>.That
|
||
is the key to this functionality.
|
||
</para>
|
||
</section>
|
||
<section xml:id="creating-a-shell">
|
||
<title>Creating a Shell</title>
|
||
<para>
|
||
Getting access to an environment often isn't enough to do real
|
||
development. Many times we need to create a
|
||
<literal>shell.nix</literal> file and do our development inside
|
||
of the environment specified by that file. This file looks a lot
|
||
like the packageing described above. The main difference is that
|
||
<literal>src</literal> points to project root and we call the
|
||
package directly.
|
||
</para>
|
||
<programlisting>
|
||
{ pkgs ? import "<nixpkgs"> {} }:
|
||
|
||
with pkgs;
|
||
|
||
let
|
||
|
||
f = { buildHex, ibrowse, jsx, erlware_commons }:
|
||
buildHex {
|
||
name = "hex2nix";
|
||
version = "0.1.0";
|
||
src = ./.;
|
||
erlangDeps = [ ibrowse jsx erlware_commons ];
|
||
};
|
||
drv = erlangPackages.callPackage f {};
|
||
|
||
in
|
||
drv
|
||
</programlisting>
|
||
<section xml:id="building-in-a-shell">
|
||
<title>Building in a shell</title>
|
||
<para>
|
||
Unfortunatly for us users of Nix, Rebar isn't very cooperative
|
||
with us from the standpoint of building a hermetic
|
||
environment. When building the rebar3 support we had to do some
|
||
sneaky things to get it not to go out and pull packages on its
|
||
own. Also unfortunately, you have to do some of the same things
|
||
when building a project inside of a Nix shell.
|
||
|
||
<orderedlist numeration="arabic">
|
||
<listitem>
|
||
<para>Run <literal>rebar3-nix-bootstrap</literal> every time
|
||
dependencies change</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Set Home to the current directory.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
|
||
If you do these two things then Rebar will be happy with you. I
|
||
codify these into a makefile. Forunately, rebar3-nix-bootstrap
|
||
is idempotent and fairly quick. so you can run it as often as
|
||
you like.
|
||
</para>
|
||
<programlisting>
|
||
# =============================================================================
|
||
# Rules
|
||
# =============================================================================
|
||
.PHONY= all test clean repl shell build test analyze bootstrap
|
||
|
||
all: test
|
||
|
||
clean:
|
||
rm -rf _build
|
||
rm -rf .cache
|
||
|
||
repl:
|
||
nix-shell --run "erl"
|
||
|
||
shell:
|
||
nix-shell --run "bash"
|
||
|
||
bootstrap:
|
||
nix-shell --pure --run "rebar3-nix-bootstrap"
|
||
|
||
build: bootstrap
|
||
nix-shell --pure --run "HOME=$(CURDIR) rebar3 compile"
|
||
|
||
analyze: bootstrap
|
||
nix-shell --pure --run "HOME=$(CURDIR) rebar3 do compile,dialyzer"
|
||
|
||
test: bootstrap
|
||
nix-shell --pure --run "HOME=$(CURDIR) rebar3 do compile,dialyzer,eunit"
|
||
|
||
</programlisting>
|
||
<para>
|
||
If you add the <literal>shell.nix</literal> as described and
|
||
user rebar as follows things should simply work.
|
||
</para>
|
||
</section>
|
||
</section>
|
||
</section>
|
||
<section xml:id="generating-packages-from-hex-with-hex2nix">
|
||
<title>Generating Packages from Hex with Hex2Nix</title>
|
||
<para>
|
||
Updating the Hex packages requires the use of the
|
||
<literal>hex2nix</literal> tool. Given the path to the Erlang
|
||
modules (usually
|
||
<literal>pkgs/development/erlang-modules</literal>). It will
|
||
happily dump a file called
|
||
<literal>hex-packages.nix</literal>. That file will contain all
|
||
the packages that use a recognized build system in Hex. However,
|
||
it can't know whether or not all those packages are buildable.
|
||
</para>
|
||
<para>
|
||
To make life easier for our users, it makes good sense to go
|
||
ahead and attempt to build all those packages and remove the
|
||
ones that don't build. To do that, simply run the command (in
|
||
the root of your <literal>nixpkgs</literal> repository). that follows.
|
||
</para>
|
||
<programlisting>
|
||
$ nix-build -A erlangPackages
|
||
</programlisting>
|
||
<para>
|
||
That will build every package in
|
||
<literal>erlangPackages</literal>. Then you can go through and
|
||
manually remove the ones that fail. Hopefully, someone will
|
||
improve <literal>hex2nix</literal> in the future to automate
|
||
that.
|
||
</para>
|
||
</section>
|
||
</chapter>
|