David Vázquez2021-07-12T21:38:22+00:00/http://davazp.net/2014/11/26/modern-library-with-asdf-and-package-inferred-systemHow to write a modern Lisp library with ASDF3 and Package Inferred System2014-11-26T00:00:00+00:002014-11-26T00:00:00+00:00David Vázquez/2014/11/26/modern-library-with-asdf-and-package-inferred-system<p>If you are a novice Lisp programmer and you are confused about the
difference between <em>systems</em> and <em>packages</em>, I recommend you to read
the <a href="http://weitz.de/packages.html" title="Packages, systems, modules, libraries - WTF?">Packages, systems, modules, libraries - WTF?</a> before you
continue.</p>
<h3 id="introduction">Introduction</h3>
<p>The most common way that people use packages nowadays is by adding a
<code class="language-plaintext highlighter-rouge">packages.lisp</code> or <code class="language-plaintext highlighter-rouge">package.lisp</code> file. This file is usually the first
file to be loaded and define the packages that the other files
use. This approach has worked for many projects. However, when
packages become larger, it requires discipline to manage the
dependencies between the files.</p>
<p>An alternative approach to the definition of packages is named <em>one
package per file</em>. As the name suggests, it consists in starting every
file with a <a href="http://clhs.lisp.se/Body/m_defpkg.htm">defpackage</a>. Because
dependencies between packages are explicit and every file has a unique
package, the dependencies between the files can be inferred.</p>
<p>This style of programming was introduced a few years ago by <em>faslpath</em>
and <em>quick-build</em>. But recently, the <em>de facto</em> standard Common Lisp
build system <a href="http://common-lisp.net/project/asdf/">ASDF3</a> added
support for it with the <em>asdf-package-system</em> extension. As a
consequence, now it is easier than ever to use it.</p>
<p>So, stay with me for the next minutes and you will learn how to use
this new approach in your projects. I hope you find it useful.</p>
<h3 id="how-to-use-it">How to use it</h3>
<p>First of all, we have to enable the <em>asdf-package-system</em> extension on
our system. We will work on a system named <code class="language-plaintext highlighter-rouge">project</code> as definition
follows</p>
<figure class="highlight"><pre><code class="language-common-lisp" data-lang="common-lisp"><span class="p">(</span><span class="nv">asdf:defsystem</span> <span class="ss">:project</span>
<span class="ss">:name</span> <span class="s">"My Project"</span>
<span class="ss">:version</span> <span class="s">"0.0.1"</span>
<span class="ss">:class</span> <span class="ss">:package-inferred-system</span>
<span class="ss">:defsystem-depends-on</span> <span class="p">(</span><span class="ss">:asdf-package-system</span><span class="p">)</span>
<span class="ss">:depends-on</span> <span class="p">(</span><span class="ss">:project/addons</span><span class="p">))</span></code></pre></figure>
<p>This defines a correspondence between systems, packages and files. A
system <code class="language-plaintext highlighter-rouge">project/foo/bar</code> refers the file <code class="language-plaintext highlighter-rouge">foo/bar.lisp</code>, which have to
provide a package <code class="language-plaintext highlighter-rouge">project/foo/bar</code>, and whose <em>used</em> and <em>imported</em>
packages refer to systems of the same name.</p>
<p>For example, as the system <code class="language-plaintext highlighter-rouge">project</code> depends on <code class="language-plaintext highlighter-rouge">project/addons</code>, the
file <code class="language-plaintext highlighter-rouge">addons.lisp</code> must be loaded first. The content of <code class="language-plaintext highlighter-rouge">addons.lisp</code>
starts with:</p>
<figure class="highlight"><pre><code class="language-common-lisp" data-lang="common-lisp"><span class="p">(</span><span class="nb">defpackage</span> <span class="ss">:project/addons</span>
<span class="p">(</span><span class="ss">:use</span> <span class="ss">:common-lisp</span> <span class="ss">:project/core</span><span class="p">)</span>
<span class="p">(</span><span class="ss">:import-from</span> <span class="ss">:cl-ppcre</span><span class="p">))</span>
<span class="p">(</span><span class="nb">in-package</span> <span class="ss">:project/addons</span><span class="p">)</span>
<span class="p">(</span><span class="nv">cl-ppcre:scan-to-strings</span> <span class="s">"h.*o"</span> <span class="p">(</span><span class="nv">hello</span><span class="p">))</span></code></pre></figure>
<p>Note that it uses the package <code class="language-plaintext highlighter-rouge">project/core</code> and “import” the
<code class="language-plaintext highlighter-rouge">cl-ppcre</code> one. Therefore, ASDF will infer that it depends on both the
<em>systems</em> <a href="http://weitz.de/cl-ppcre/">cl-ppcre</a> and <code class="language-plaintext highlighter-rouge">project/core</code>, so
they must be loaded first. But remember, the system <code class="language-plaintext highlighter-rouge">project/core</code>
refers to the file <code class="language-plaintext highlighter-rouge">core.lisp</code>:</p>
<figure class="highlight"><pre><code class="language-common-lisp" data-lang="common-lisp"><span class="p">(</span><span class="nb">defpackage</span> <span class="ss">:project/core</span>
<span class="p">(</span><span class="ss">:use</span> <span class="ss">:common-lisp</span><span class="p">)</span>
<span class="p">(</span><span class="ss">:export</span> <span class="ss">#:hello</span><span class="p">))</span>
<span class="p">(</span><span class="nb">in-package</span> <span class="ss">:project/core</span><span class="p">)</span>
<span class="p">(</span><span class="nb">defun</span> <span class="nv">hello</span> <span class="p">()</span>
<span class="s">"Hello!"</span><span class="p">)</span></code></pre></figure>
<p>And that is all. This file has no external dependencies. Then, if we
try to load our system</p>
<div class="language-common-lisp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="nv">asdf:load-system</span> <span class="s">"project"</span><span class="p">)</span>
</code></pre></div></div>
<p>the systems and files <code class="language-plaintext highlighter-rouge">core.lisp</code>, <code class="language-plaintext highlighter-rouge">cl-ppcre</code> and <code class="language-plaintext highlighter-rouge">addons.lisp</code> will
be loaded in the proper order.</p>
<h3 id="integration-with-other-systems">Integration with other systems</h3>
<p>What if we use a package but the system that provides such a package
has not the same (downcased) name? For example, the system
<a href="http://common-lisp.net/project/closer/closer-mop.html">closer-mop</a>
provides a package named <em>c2cl</em>. In order to let ASDF know how to find
the system for a package, we can call the function
<code class="language-plaintext highlighter-rouge">register-system-packages</code> with the name of the systems and the
packages it provides as argument. In our case, we would include the
following in our <code class="language-plaintext highlighter-rouge">project.asd</code>:</p>
<div class="language-common-lisp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="nv">register-system-packages</span> <span class="ss">:closer-mop</span> <span class="o">'</span><span class="p">(</span><span class="ss">:c2cl</span><span class="p">))</span>
</code></pre></div></div>
<h3 id="a-last-trick">A Last Trick</h3>
<p>Most of the time we want to <em>export</em> a single package with all the
symbols from the “subpackages”. This can be done very easily if you
use the <em>UIOP</em> library shipped with ASDF. For example, let us define a
file named <code class="language-plaintext highlighter-rouge">all.lisp</code> in our example like:</p>
<figure class="highlight"><pre><code class="language-common-lisp" data-lang="common-lisp"><span class="p">(</span><span class="nv">uiop/package:define-package</span> <span class="ss">:project/all</span>
<span class="p">(</span><span class="ss">:nickname</span> <span class="ss">:project</span><span class="p">)</span>
<span class="p">(</span><span class="ss">:use-reexport</span> <span class="ss">:project/core</span> <span class="ss">:project/addons</span><span class="p">))</span></code></pre></figure>
<p>Now, the system <code class="language-plaintext highlighter-rouge">project/all</code> depends on both <code class="language-plaintext highlighter-rouge">project/core</code> and
<code class="language-plaintext highlighter-rouge">project/addons</code>, and it will reexport the symbols into a <code class="language-plaintext highlighter-rouge">:project</code>
package.</p>
<h3 id="futher-information">Futher information</h3>
<p>If you want to know more about ASDF3 and package-system:</p>
<ul>
<li>
<p><a href="http://common-lisp.net/project/asdf/asdf/The-package_002dinferred_002dsystem-extension.html#The-package_002dinferred_002dsystem-extension">ASDF Manual. The package-inferred-system extension</a>.</p>
</li>
<li>
<p>ASDF 3, or Why Lisp is Now an Acceptable Scripting Language<br />
(Extended version) by François-René Rideau.<br />
<a href="http://fare.tunes.org/files/asdf3/asdf3-2014.html">http://fare.tunes.org/files/asdf3/asdf3-2014.html</a></p>
</li>
</ul>
http://davazp.net/2013/07/16/Low-level-PostgreSQL-access-from-LispPostgreSQL from Lisp2013-07-16T00:00:00+00:002013-07-16T00:00:00+00:00David Vázquez/2013/07/16/Low-level-PostgreSQL-access-from-Lisp<p>Hello everybody,</p>
<p>It has been a long time before my last post. Many things happened: I
had a great time in <a href="http://weitz.de/eclm2013/">ECL2013</a>, I
<a href="https://github.com/davazp/jscl/blob/master/src/compiler/codegen.lisp">wrote a code generator for JSCL</a>,
exams in the university… and suffering the heat at home in Spain, of
course.</p>
<p>These days, at work, I am migrating our old SQL Server to PostgreSQL.
We used to use CLSQL to access to the server from Lisp-side, but in a
very straightforward way. We built everything on <code class="language-plaintext highlighter-rouge">query</code>. Although
CLSQL is a big monster, it misses parametric queries, and you have to
escape arguments yourself. Indeed, it was not as much stable as I
would like. But let apart exotic features.</p>
<p>Therefore I gave a try to other libraries. The obvious choice would be
<a href="http://marijnhaverbeke.nl/postmodern/">Postmodern</a>, but it failed
soon. DAO capabilities seem incomplete and, I think, they should be
other library, but I can live ignoring it. However, have a look to
<a href="http://marijnhaverbeke.nl/postmodern/postmodern.html#query">query</a>:
it has a strange arg/format mix and it is a macro! It means you will
fail to build simple abstractions on it.</p>
<p>Fortunately, Postmodern’s author created a lower library
<a href="http://marijnhaverbeke.nl/postmodern/cl-postgres.html">cl-postgres</a>,
which I can suit easily:</p>
<div class="language-common-lisp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="nb">defun</span> <span class="nv">query</span> <span class="p">(</span><span class="nv">query</span> <span class="k">&rest</span> <span class="nv">args</span><span class="p">)</span>
<span class="p">(</span><span class="nv">cl-postgres:prepare-query</span> <span class="vg">*database*</span> <span class="s">""</span> <span class="nv">query</span><span class="p">)</span>
<span class="p">(</span><span class="nv">cl-postgres:exec-prepared</span> <span class="vg">*database*</span> <span class="s">""</span> <span class="nv">args</span> <span class="ss">'cl-postgres:list-row-reader</span><span class="p">))</span>
</code></pre></div></div>
<p>but of course I will have to write connection pooling and other
features myself, or perhaps mix postmodern and cl-postgres carefully.</p>
<p>In conclusion, I think we should be a little bit more <em>conventional</em>
in the abstractions we build, and not to forget intermediate
abstractions.</p>
http://davazp.net/2013/05/01/the-surprising-jscl-risingThe surprising JSCL rising2013-05-01T00:00:00+00:002013-05-01T00:00:00+00:00David Vázquez/2013/05/01/the-surprising-jscl-rising<p>I am just back from a few days of holidays and have seen how active
JSCL is! It is really exciting.</p>
<p>This rising is due to <a href="https://github.com/abeaumont">abeaumont</a>
probably. I stopped working, but he came up with an idea I liked. It
is, why don’t integrate JSCL into <a href="http://conkeror.org/">Conkeror</a>?
We would end up with a Lisp programmable browser… that’s a practical
reason! We did some tests locally and it seemed to work, but now it
is time for improving JSCL itself.</p>
<p>It is not the only platform where JSCL could run on. I plan to add
support for <a href="http://nodejs.org/">Node JS</a>. Indeed, it will let us use
the REPL and tests in the terminal in addition to the browser, which
is a much more convenient way to program.</p>
<p>In the meantime, I would like to improve the compiler a little bit.</p>
<p>Suggestion and patches are very welcome if you want to join us.</p>
<p><a href="https://github.com/davazp/jscl">https://github.com/davazp/jscl</a></p>
http://davazp.net/2013/01/24/Special-argumentsSpecial arguments2013-01-24T00:00:00+00:002013-01-24T00:00:00+00:00David Vázquez/2013/01/24/Special-arguments<p>I have learnt a new trick these days. Did you know that you can
declare an argument as a special variable? Quoting <a href="http://www.lispworks.com/documentation/HyperSpec/Body/03_daa.htm">Hyperspec</a>:</p>
<blockquote>
<p>Each required parameter is specified by a parameter variable var. var
is bound as a lexical variable <em>unless it is declared special</em>.</p>
</blockquote>
<p>It seems to work also for optional and keyword arguments, but
Hyperspec does not mention it explicitly. For example, yo can do:</p>
<div class="language-common-lisp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="nb">defvar</span> <span class="vg">*variable*</span><span class="p">)</span>
<span class="p">(</span><span class="nb">defun</span> <span class="nv">foo</span> <span class="p">()</span> <span class="vg">*variable*</span><span class="p">)</span>
<span class="p">(</span><span class="nb">defun</span> <span class="nv">foobar</span> <span class="p">(</span><span class="k">&optional</span> <span class="vg">*variable*</span><span class="p">)</span>
<span class="p">(</span><span class="nv">foo</span><span class="p">))</span>
</code></pre></div></div>
http://davazp.net/2013/01/04/lisp-to-javascriptLisp to Javascript compiler2013-01-04T00:00:00+00:002013-01-04T00:00:00+00:00David Vázquez/2013/01/04/lisp-to-javascript<hr />
<p><strong>UPDATE 22 Nov 2014: The project was renamed to <em>JSCL</em> and now is hosted at <a href="http://github.com/davazp/jscl">http://github.com/davazp/jscl</a>.
You can also try <a href="http://davazp.net/jscl/jscl.html">the live demo</a>.</strong></p>
<hr />
<p><a href="http://puntoblogspot.blogspot.ie/">Raimon Grau</a> and I have been
working on a simple Lisp to Javascript compiler, in order to better
understand how to bootstrap it. Now we got it. It compiles itself and
the resulting code can be used from the browser. It can even call its
own internals, which is quite nice.</p>
<p>It is not very helpful, you can
<a href="http://davazp.github.com/ecmalisp/ecmalisp.html">try it</a>. Now I have
to think if it could be useful for something… maybe a live Lisp
tutorial, or make it suitable for Android. Any suggestions?</p>
<p>The repository with the whole history is at:</p>
<p><a href="http://github.com/davazp/ecmalisp/">http://github.com/davazp/ecmalisp/</a></p>
<p><a href="http://puntoblogspot.blogspot.ie/">Raimon Grau</a> and I have been
working on a simple Lisp to Javascript compiler, in order to better
understand how to bootstrap it. Now we got it. It compiles itself and
the resulting code can be used from the browser. It can even call its
own internals, which is quite nice.</p>
<p>It is not very helpful, you can
<a href="http://davazp.github.com/ecmalisp/ecmalisp.html">try it</a>. Now I have
to think if it could be useful for something… maybe a live Lisp
tutorial, or make it suitable for Android. Any suggestions?</p>
<p>The repository with the whole history is at:</p>
<p><a href="http://github.com/davazp/ecmalisp/">http://github.com/davazp/ecmalisp/</a></p>