Johns Bloghttp://www.johnstowers.co.nz/blog/2015-07-27T17:05:00+02:00Installing OpenCV 3 on Ubuntu 12.042015-07-27T17:05:00+02:00John Stowerstag:www.johnstowers.co.nz/blog,2015-07-27:2015/07/27/opencv3-ubuntu1204/<p>After spending the morning fighting my way through this, here are some
notes for myself and Google to index. Before you get started;</p>
<ul class="simple">
<li>Install a slightly newer cmake</li>
</ul>
<div class="highlight"><pre><span></span>sudo add-apt-repository --yes ppa:kalakris/cmake
sudo apt-get update -qq
sudo apt-get install cmake
</pre></div>
<ul class="simple">
<li>Set up a lightweight development environment (or similar) as per <a class="reference external" href="http://www.johnstowers.co.nz/blog/2013/09/29/lightweight-development-prefixes/">this article</a></li>
<li>Download and install <a class="reference external" href="https://libav.org/download/">Libav v9</a></li>
</ul>
<div class="highlight"><pre><span></span>./configure --prefix<span class="o">=</span>/opt/opencv3 --enable-pic --disable-static --enable-shared
</pre></div>
<ul class="simple">
<li>Git clone the <a class="reference external" href="https://github.com/Itseez/opencv_contrib">opencv_contrib repository</a></li>
<li>Check out the <tt class="docutils literal">3.0.0</tt> tag</li>
<li>Download and unpack the <a class="reference external" href="http://opencv.org/opencv-3-0.html">OpenCV 3.0 release</a></li>
<li>Create a build directory</li>
</ul>
<div class="highlight"><pre><span></span><span class="nb">cd</span> opencv-3.0.0
rm -rf ./build <span class="o">&&</span> mkdir build <span class="o">&&</span> <span class="nb">cd</span> build
</pre></div>
<ul class="simple">
<li>Invoke the build like follow</li>
</ul>
<div class="highlight"><pre><span></span>cmake -D <span class="nv">CMAKE_BUILD_TYPE</span><span class="o">=</span>RELEASE <span class="se">\</span>
-D <span class="nv">CMAKE_INSTALL_PREFIX</span><span class="o">=</span>/opt/opencv3 <span class="se">\</span>
-D <span class="nv">PYTHON_INCLUDE_DIR</span><span class="o">=</span><span class="k">$(</span>python -c <span class="s2">"from distutils.sysconfig import get_python_inc; print(get_python_inc())"</span><span class="k">)</span> <span class="se">\</span>
-D <span class="nv">PYTHON_EXECUTABLE</span><span class="o">=</span><span class="k">$(</span>which python<span class="k">)</span> <span class="se">\</span>
-D <span class="nv">PYTHON_PACKAGES_PATH</span><span class="o">=</span><span class="k">$(</span>python -c <span class="s2">"from distutils.sysconfig import get_python_lib; print(get_python_lib())"</span><span class="k">)</span> <span class="se">\</span>
-D <span class="nv">WITH_VTK</span><span class="o">=</span>OFF -D <span class="nv">WITH_CUDA</span><span class="o">=</span>OFF <span class="se">\</span>
-D <span class="nv">BUILD_PERF_TESTS</span><span class="o">=</span>OFF -D <span class="nv">BUILD_TESTS</span><span class="o">=</span>OFF -D <span class="nv">WITH_FFMPEG</span><span class="o">=</span>ON -D <span class="nv">WITH_1394</span><span class="o">=</span>OFF <span class="se">\</span>
-D <span class="nv">WITH_WEBP</span><span class="o">=</span>OFF -D <span class="nv">ENABLE_SSE42</span><span class="o">=</span>ON -D <span class="nv">ENABLE_SSE41</span><span class="o">=</span>ON -D <span class="nv">ENABLE_SSE42</span><span class="o">=</span>ON <span class="se">\</span>
-D <span class="nv">ENABLE_POPCNT</span><span class="o">=</span>ON -D <span class="nv">ENABLE_AVX</span><span class="o">=</span>ON -D <span class="nv">ENABLE_SSSE3</span><span class="o">=</span>ON <span class="se">\</span>
-D <span class="nv">INSTALL_PYTHON_EXAMPLES</span><span class="o">=</span>ON -D <span class="nv">WITH_GSTREAMER</span><span class="o">=</span>OFF <span class="se">\</span>
-D <span class="nv">WITH_GTK_2_X</span><span class="o">=</span>ON -D <span class="nv">INSTALL_C_EXAMPLES</span><span class="o">=</span>ON <span class="se">\</span>
-D <span class="nv">BUILD_EXAMPLES</span><span class="o">=</span>ON <span class="se">\</span>
-D <span class="nv">OPENCV_EXTRA_MODULES_PATH</span><span class="o">=</span>../opencv_contrib/modules/ <span class="se">\</span>
..
</pre></div>
<p>Some other observations;</p>
<ul class="simple">
<li>If the build doesn't succeed you need to remove and recreate the build
directory again (as CMake somehow fails to cache the Python configuration).</li>
<li>The <tt class="docutils literal"><span class="pre">gtk+-3</span></tt> gui backend doesn't auto-resize image windows correctly</li>
<li>It would probably be easier to move to a newer Ubuntu LTS, or preferably
debian stable.</li>
</ul>
FlyMAD - The Fly Mind Altering Device2014-05-27T21:05:00+02:00John Stowerstag:www.johnstowers.co.nz/blog,2014-05-27:2014/05/27/flymad/<p>Today I'm proud to announce the availability of <a class="reference external" href="https://github.com/strawlab/flymad">all source code</a>,
and the <a class="reference external" href="http://www.nature.com/doifinder/10.1038/nmeth.2973">advanced online publication</a>
of our paper</p>
<blockquote>
Bath DE*, Stowers JR*, Hörmann D, Poehlmann A, Dickson BJ, Straw AD (* equal contribution) (2014)
<strong>FlyMAD: Rapid thermogenetic control of neuronal activity in freely-walking Drosophila.</strong>
<a class="reference external" href="http://www.nature.com/doifinder/10.1038/nmeth.2973">Nature Methods.</a> doi 10.1038/nmeth.2973</blockquote>
<p><a class="reference external" href="http://flymad.strawlab.org">FlyMAD</a> (Fly Mind Altering Device) is a system for targeting freely walking
flies (Drosophila) with lasers. This allows rapid thermo- and opto- genetic manipulation of the
fly nervous system in order to study neuronal function.</p>
<a class="reference external image-reference" href="http://www.johnstowers.co.nz/blog/static/images/strawlab/flymad_intro.png"><img alt="|filename|images/strawlab/flymad_intro_sml.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/flymad_intro_sml.png" /></a>
<p>The scientific aspects of the publication are better summarised on
<a class="reference external" href="http://www.nature.com/news/laser-beam-makes-flies-flirt-1.14794">nature.com</a>,
<a class="reference external" href="http://www.sciencedaily.com/releases/2014/05/140525154744.htm">here</a>,
on our <a class="reference external" href="http://strawlab.org/2014/05/25/flymad/">laboratory website</a>, or
in the video at the bottom of this post.</p>
<p>Briefly however; if one wishes to link function to specific neurons one could conceive of
two broad approaches. First, observe the firing of the neurons in
<a class="reference external" href="http://www.imp.ac.at/news/press-releases/press-release/press-release-high-speed-imaging-method-captures-entire-brain-activity/">real time</a>
using fluorescence or other microscopy techniques. Second, use genetic techniques to engineer organisms with
light or temperature sensitive proteins bound to specific neuronal classes such that by the application
of heat or light, activity in those neurons can be modulated.</p>
<p>Our system takes the second approach; our innovation being that by using real time computer vision and
control techniques we are able to track freely walking Drosophila and apply precise (sub 0.2mm)
opto- or thermogenetic stimulation to study the role of specific neurons in a diverse array of behaviours.</p>
<p>This blog post will cover a few of the technical and architectural decisions I made in the creation of the system.
Perhaps it is easiest to start with a screenshot and schematic of the system in operation</p>
<a class="reference external image-reference" href="http://www.johnstowers.co.nz/blog/static/images/strawlab/flymad_screenshot.png"><img alt="|filename|images/strawlab/flymad_screenshot_sml.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/flymad_screenshot_sml.png" /></a>
<p>Here one can see two windows showing images from the two tracking cameras, associated image processing configuration parameters
(and their results, at 120fps). In the center at the bottom is visible the ROS based experimental control UI.
Schematically, the two cameras and lasers are arranged like the following</p>
<a class="reference external image-reference" href="http://www.johnstowers.co.nz/blog/static/images/strawlab/render2.png"><img alt="|filename|images/strawlab/render2_sml.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/render2_sml.png" /></a>
<p>In this image you can also see the thorlabs 2D galvanometers (top left), and the dichroic mirror
which allows aligning the camera and laser on the same optical axis.</p>
<p>By pointing the laser at flies freely walking in the arena below, one can subsequently
deliver heat or light to specific body regions.</p>
<div class="section" id="general-architecture">
<h2>General Architecture</h2>
<p>The system consists of hardware and software elements. A small microcontroller and
digital to analogue converter generate analog control signals to point the
2D galvanometers and to control laser power. The device communicates with the host
PC over a serial link. There are two cameras in the system; a wide camera for fly position
tracking, and a second high magnification camera for targeting specific regions of the fly.
This second camera is aligned with the laser beam, and its view can be pointed
anywhere in the arena by the galvanometers.</p>
<p>The software is conceptually three parts; image processing code, tracking and targeting code, and
experimental logic. All software elements communicate using robot operating
system (ROS) interprocess communication layer. The great majority of
code is written in python.</p>
<img alt="|filename|images/strawlab/path8510_sml.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/path8510_sml.png" />
</div>
<div class="section" id="robot-operating-system-ros">
<h2>Robot Operating System (ROS)</h2>
<p><a class="reference external" href="http://www.ros.org">ROS</a> is a framework traditionally used for building
complex robotic systems. In particular it has a relatively good performance and
simple, strongly typed, inter-process-communication framework and serialization format.</p>
<p>Through its (pure) python interface one can build a complex system of multiple
processes who communicate (primarily) by publishing and subscribing to
message "topics". An example of the ROS processes running during a FlyMAD
experiment can be seen below.</p>
<a class="reference external image-reference" href="http://www.johnstowers.co.nz/blog/static/images/strawlab/s3_crop.png"><img alt="|filename|images/strawlab/s3_crop_sml.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/s3_crop_sml.png" /></a>
<p>The lines connecting the nodes represent the flow of information across the
network, and all messages can be simultaneously recorded (see <tt class="docutils literal">/recorder</tt>)
for analysis later. Furthermore, the isolation of the individual processes
improves robustness and defers some of the responsibility for realtime
performance from myself / Python, to the Kernel and to my overall
architecture.</p>
<blockquote>
For more details on ROS and on why I believe it is a good tool for
creating reliable reproducible science, see my
<a class="reference external" href="http://johnstowers.co.nz/blog/2013/10/11/ros-freeze/">previous post</a>,
my <a class="reference external" href="http://www.youtube.com/watch?v=marMd_K8Z1M">Scipy2013 video</a> and
<a class="reference external" href="https://speakerdeck.com/nzjrs/managing-complex-experiments-automation-and-analysis-using-robot-operating-system">presentation</a></blockquote>
</div>
<div class="section" id="image-processing">
<h2>Image Processing</h2>
<p>There are two image processing tasks in the system. Both are implemented as
<a class="reference external" href="http://code.astraw.com/projects/motmot/fview.html">FView</a>
plugins and communicate with the rest of the system using ROS.</p>
<p>Firstly, the position of the fly (flies) in the arena, as seen by the
wide camera, must be determined. Here, a simple threshold approach is used to
find candidate points and image moments around those points are used to find the
center and slope of the fly body. A lookup table is used to point the
galvanometers in an open-loop fashion approximately at the fly.</p>
<p>With the fly now located in the field of view of the high magnification camera a
second real time control loop is initiated. Here, the fly body or head is detected,
and a closed loop PID controller finely adjusts the galvanometer position to achieve
maximum targeting accuracy. The accuracy of this through the mirror (TTM) system asymptotically
approaches 200μm and at 50 msec from onset the accuracy of head detection is 400 ± 200 μm.
From onset of TTM mode, considering other latencies in the system (gigabit ethernet, 5 ms,
USB delay, 4 ms, galvanometer response time, 7 ms, image processing 8ms, and image
acquisition time, 5-13 ms) total 32 ms, this shows the real time
targeting stabilises after 2-3 frames and comfortably operates at better than 120 frames
per second.</p>
<a class="reference external image-reference" href="http://www.johnstowers.co.nz/blog/static/images/strawlab/s1_crop.png"><img alt="|filename|images/strawlab/s1_crop_sml.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/s1_crop_sml.png" /></a>
<p>To reliably track freely walking flies, the head and body step image processing
operations must take less than 8ms. Somewhat frustratingly, a traditional template
matching strategy worked best. On the binarized, filtered image, the largest contour
is detected (c, red). Using an ellipse fit to the contour points (c,green), the contour
is rotated into an upright orientation (d). A template of the fly (e) is compared with the
fly in both orientations and the best match is taken.</p>
<p>I mention the template strategy as being disappointing only because I spent considerable time
evaluating newer, shinier, feature based approaches and could not achieve the closed loop
performance I needed. While the newer descriptors, BRISK, FREAK, ORB were faster than the previous
class, nether (in total) were significantly more reliable considering changes in illumination than
SURF - which could not meet the <8ms deadline reliably. I also spent considerable time testing
edge based (binary) descriptors such as edgelets, or edge based (gradient) approaches such as
dominant orientation templates or gradient response maps. The most promising of this class was local
shape context descriptors, but I also could not get the runtime below 8ms. Furthermore, one advantage
of the contour based template matching strategy I implemented, was that graceful degradation was
possible - should a template match not be found (which occurred in <1% of frames), an estimate
of the centre of mass of the fly was still present, which still allowed degraded targeting performance.
No such graceful fallback was possible using feature correspondence based strategies.</p>
<p>There are two implementations of the template match operation - GPU and CPU based. The CPU matcher
uses the python OpenCV bindings (and numpy in places), the GPU matcher uses cython to wrap a small
c++ library that does the same thing using OpenCV 2.4 Cuda GPU support (which is not otherwise
accessible from python). Intelligently, the python OpenCV bindings use numpy arrays to store
image data, so passing data from Python to native code is trivial and efficient.</p>
<blockquote>
I also gave <a class="reference external" href="https://speakerdeck.com/nzjrs/interfacing-with-native-code-from-python">a presentation</a>
comparing different strategies of interfacing python with native code. The
<a class="reference external" href="https://github.com/strawlab/py4science-vbc/tree/master/2012-04-13">provided source code</a>
includes examples using python/ctypes/cython/numpy and permutations thereof.</blockquote>
<p>The GPU code-path is only necessary / beneficial for very large templates and
higher resolution cameras (as used by our collaborator) and in general the CPU
implementation is used.</p>
</div>
<div class="section" id="experimental-control-gui">
<h2>Experimental Control GUI</h2>
<p>To make FlyMAD easier to manage and use for biologists I wrote a small GUI using
Gtk (PyGObject), and my ROS utility GUI library
<a class="reference external" href="https://github.com/strawlab/rosgobject">rosgobject</a>.</p>
<a class="reference external image-reference" href="http://www.johnstowers.co.nz/blog/static/images/strawlab/gflymad.png"><img alt="|filename|images/strawlab/gflymad_sml.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/gflymad_sml.png" /></a>
<p>On the left you can see buttons for launching individual ROS nodes. On the right
are widgets for adjusting the image processing and control parameters (these
widgets display and set ROS parameters). At the bottom are realtime statistics showing
the TTM image processing performance (as published to ROS topics).</p>
<p>Like good ROS practice, once reliable values are found for all adjustable parameters
they can be recorded in a <tt class="docutils literal">roslaunch</tt> file allowing the whole system to
be started with known configuration from a single command.</p>
</div>
<div class="section" id="manual-scoring-of-videos">
<h2>Manual Scoring of Videos</h2>
<p>For certain experiments (such as courtship) videos recorded during the experiment
must be watched and behaviours must be manually annotated. To my surprise, no tools
exist to make this relatively common behavioural neuroscience task any easier
(and easier matters; it is not uncommon to score 10s to 100s of hours of videos).</p>
<p>During every experiment, <a class="reference external" href="http://code.astraw.com/projects/motmot/fly-movie-format.html">RAW uncompressed videos</a>
from both cameras are written to disk (uncompressed videos are chosen for performance reasons, because
SSDs are cheap, and because each frame can be precisely timestamped).
Additionally, <tt class="docutils literal">rosbag</tt> files record the complete state of the experiment at
every instant in time (as described by all messages passing
between ROS nodes). After each experiment finishes, the uncompressed videos from
each camera are composited together, along with metadata such as the frame
timestamp, and a h264 encoded mp4 video is created for scoring.</p>
<p>After completing a full day of experiments one can then score / annotate
videos in bulk. The scorer is written in Python, uses Gtk+ and PyGObject for the
UI, and <a class="reference external" href="https://wiki.videolan.org/Python_bindings">vlc.py</a> for decoding the
video (I chose vlc due to the lack of working gstreamer PyGObject support on Ubuntu
12.04)</p>
<a class="reference external image-reference" href="http://www.johnstowers.co.nz/blog/static/images/strawlab/scorer.png"><img alt="|filename|images/strawlab/scorer_sml.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/scorer_sml.png" /></a>
<p>In addition to allowing play, pause and single frame scrubbing through the video,
pressing any of <tt class="docutils literal">qw,as,zx,cv</tt> pairs of keys indicates that a
a behaviour has started or finished. At this instant the current video frame is
extracted from the video, and optical-character-recognition is performed on the
top left region of the frame in order to extract the timestamp. When the video
is finished, a <a class="reference external" href="http://pandas.pydata.org/">pandas dataframe</a> is created which contains
all original experimental <tt class="docutils literal">rosbag</tt> data, and the manually annotated behaviour
against on a common timebase.</p>
</div>
<div class="section" id="distributing-complex-experimental-software">
<h2>Distributing complex experimental software</h2>
<p>The system was not only run by myself, but by <a class="reference external" href="http://janelia.org/lab/dickson-lab">collaborators</a>,
and we hope in future, by others too. To make this possible we generate a single
file self installing executable using <a class="reference external" href="https://github.com/megastep/makeself">makeself</a>,
and we only officially support one distribution - Ubuntu 12.04 LTS and x86_64.</p>
<p>The <tt class="docutils literal">makeself</tt> installer performs the following steps</p>
<ol class="arabic simple">
<li>Adds our Debian repository to the system</li>
<li>Adds the official ROS Debian repository to the system</li>
<li>Adds our custom ROS stacks (FlyMAD from tarball and rosgobject from git)
to the ROS environment</li>
<li>Calls <tt class="docutils literal">rosmake flymad</tt> to install all system dependencies and build
and non-binary ROS packages.</li>
<li>Creates a FlyMAD desktop file to start the software easily</li>
</ol>
<p>We also include a version check utility in the FlyMAD GUI which notifies the user
when a newer version of the software is available.</p>
</div>
<div class="section" id="the-results">
<h2>The Results</h2>
<p>Using FlyMAD and the architecture I have described above we created a novel system
to perform temporally and spatially precise opto and thermogenetic activation
of freely moving drosophila. To validate the system we showed distinct timing
relationships for two neuronal cell types previously linked to courtship song, and
demonstrated compatibility of the system to visual behaviour experiments.</p>
<p>Practically we were able to develop and simultaneously operate this complex
real-time assay in two countries. The system was conceived and built in approximately
one year using Python. FlyMAD utilises many best-in-class libraries and frameworks
in order to meet the demanding real time requirements (OpenCV, numpy, ROS).</p>
<p>We are proud to make the entire system available to the Drosophila community
under an open source license, and we look forward to its adoption by our peers.</p>
<p>For those still reading, I encourage you to view the supplementary video below,
where its operation can be seen.</p>
<iframe width="670" height="361" src="http://www.youtube.com/embed/SDIEBgtSSJk?rel=0" frameborder="0" allowfullscreen></iframe><p>Comments, suggestions or corrections can be <a class="reference external" href="http://www.johnstowers.co.nz/blog/pages/about-me.html">emailed to me</a>
or left on <a class="reference external" href="https://plus.google.com/113069881945587451927/posts/fPQn2GMmsbC">Google Plus</a></p>
</div>
Distributing Pure Python ROS Applications2013-10-11T00:46:32+02:00John Stowerstag:www.johnstowers.co.nz/blog,2013-10-11:2013/10/11/ros-freeze/<p>In June 2013 I was lucky to speak at the fantastic SciPy2013 conference
(scientific computing with python). I spoke about a work flow and tools we have
developed at <a class="reference external" href="http://www.strawlab.org">strawlab</a>. The title of my talk
was <strong>Managing Complex Experiments, Automation, and
Analysis using Robot Operating System</strong>.
The <a class="reference external" href="http://www.youtube.com/watch?v=marMd_K8Z1M">video</a> of that
talk is included below;</p>
<iframe width="670" height="361" src="http://www.youtube.com/embed/marMd_K8Z1M?rel=0" frameborder="0" allowfullscreen></iframe><p>And here are the <a class="reference external" href="https://speakerdeck.com/nzjrs/managing-complex-experiments-automation-and-analysis-using-robot-operating-system">accompanying slides</a>;</p>
<script async class="speakerdeck-embed" data-id="6ae4eb60c4690130f2271247c9814889" data-ratio="1.33333333333333" src="http://speakerdeck.com/assets/embed.js"></script><p>This post describes a tool I developed for
distributing ROS packages to scientific collaborators. That software is
called <a class="reference external" href="https://github.com/strawlab/ros-freeze">ros-freeze</a>.</p>
<p>For those of you not aware, <a class="reference external" href="http://www.ros.org">ROS</a> is a great framework
traditionally targeted for robotics but usable in other fields too. In particular
it has a relatively good performance and simple, strongly typed, inter-process-communication
framework and serialization format. This is simultaneously useful for creating
distributed realtime-ish systems with comprehensive logging of the system
state. Best of all, the python interface to ROS is very clean.</p>
<img alt="|filename|images/strawlab/rospyramid.png" class="align-center" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/rospyramid.png" />
<p>Unfortunately, being a framework, ROS is rather all-or-nothing (going as
far as to describe itself as a meta-operating system). The basic ROS
install is several gigabytes, and building it yourself can be rather
difficult. Furthermore, as I mentioned in my presentation, it is attractive
to use the built in ROS tool <tt class="docutils literal">rosbag</tt> for recording timestamped data
to disk. Unfortunately, reading these files again needs ROS,
thus necessarily coupling experimental data to the software used
to collect it.</p>
<p>To remedy this I wrote <a class="reference external" href="https://github.com/strawlab/ros-freeze">ros-freeze</a>,
a python tool to convert any ROS package into a pure-python package including
all of the dependencies. Collaborators can then install the python
package and immediately have access to all the same ROS packages and libraries
without having to build the whole ROS stack.</p>
<div class="section" id="converting-your-ros-package">
<h2>Converting your ROS package</h2>
<ol class="arabic">
<li><p class="first">Download ros-freeze from <a class="reference external" href="https://github.com/strawlab/ros-freeze">here</a></p>
</li>
<li><p class="first">Modify setup-freeze.py according to your needs</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">setuptools</span> <span class="kn">import</span> <span class="n">setup</span>
<span class="kn">from</span> <span class="nn">rosfreeze</span> <span class="kn">import</span> <span class="n">import_ros_package</span><span class="p">,</span> <span class="n">get_disutils_cmds</span>
<span class="n">MY_PACKAGE</span> <span class="o">=</span> <span class="s1">'foo'</span>
<span class="n">setup</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">'python-ros-</span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">MY_PACKAGE</span><span class="p">,</span>
<span class="n">version</span><span class="o">=</span><span class="s1">'1.0'</span><span class="p">,</span>
<span class="n">description</span><span class="o">=</span><span class="s1">'Pure Python ROS </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">MY_PACKAGE</span><span class="p">,</span>
<span class="n">author</span><span class="o">=</span><span class="s1">''</span><span class="p">,</span>
<span class="n">author_email</span><span class="o">=</span><span class="s1">''</span><span class="p">,</span>
<span class="o">**</span><span class="n">get_disutils_cmds</span><span class="p">(</span><span class="n">srcdir</span><span class="p">,</span> <span class="n">bindir</span><span class="p">,</span> <span class="n">datadir</span><span class="p">)</span>
<span class="p">)</span>
</pre></div>
</li>
<li><p class="first">Build a python egg (for example)</p>
<p><tt class="docutils literal">$ python <span class="pre">setup-freeze.py</span> bdist_egg</tt></p>
</li>
<li><p class="first">Install that egg into your virtual environment</p>
</li>
</ol>
</div>
<div class="section" id="caveats-and-other-notes">
<h2>Caveats and other Notes</h2>
<ul class="simple">
<li>this is currently working on ROS Electric (an old release, at work
we have chosen to stick with Ubuntu 12.04LTS)</li>
<li>changes for other ROS distributions might be necessary, so please get in touch</li>
<li>this is successfully tested on ROS packages containing tens of thousands of lines of
code and dozens of ROS dependencies.</li>
<li>although recent ROS releases have improved the package management situation (by embracing
deb packaging, yay!) this tool provides an unprecedentedly easy way to distribute your
pure python ROS applications</li>
</ul>
</div>
<div class="section" id="a-pure-python-ros-distribution">
<h2>A Pure Python ROS Distribution?</h2>
<p>One side effect of this was the packaging of the pure python ROS core as an easily installable
python egg. This means that you can write, debug and test python ROS nodes without having to
install the whole ROS distribution.</p>
<p>One can even go as far as running <tt class="docutils literal">rosmaster</tt> and the command line tools
(<tt class="docutils literal">rosnode</tt>, <tt class="docutils literal">rosparam</tt>, etc)!</p>
<p>You can download the <a class="reference external" href="https://github.com/strawlab/ros-freeze/releases">python-ros-electric package from here</a>.</p>
</div>
Lightweight Development Prefixes2013-09-29T18:05:00+02:00John Stowerstag:www.johnstowers.co.nz/blog,2013-09-29:2013/09/29/lightweight-development-prefixes/<p>Lately I have been writing a lot of native code against multiple OpenCV
versions. Like many Linux developers I tend to keep different development prefixes
isolated using LD_LIBRARY_PATH (and friends).</p>
<p>I recently took the time to clean up a little script I use for this
purpose, and posted it online. Inspired by virtualenv, it now has a prompt!</p>
<img alt="|filename|images/devenv-prompt.png" src="http://www.johnstowers.co.nz/blog/static/images/devenv-prompt.png" />
<p>To use the script:</p>
<ol class="arabic simple">
<li>create the development directory</li>
<li>copy the script there</li>
<li><code>source /path/to/env.sh</code></li>
</ol>
<p>please leave comments or suggestions on the <a class="reference external" href="https://gist.github.com/nzjrs/6268729/">gist</a>.</p>
<script src="https://gist.github.com/nzjrs/6268729.js"></script>GNOME Tweak Tool 3.10 Improvements2013-09-24T12:25:32+02:00John Stowerstag:www.johnstowers.co.nz/blog,2013-09-24:2013/09/24/tweak-tool-310/<p>In collaboration with GSOC student <a href="http://alexmudoz.github.io/">Alex Muñoz</a>
and designer <a href="http://afaikblog.wordpress.com/">Allan Day</a>, GNOME Tweak Tool has
seen many improvements this cycle, both 'under the hood', and most noticeably,
in the form of a modern GNOME3 UI
<a href="https://github.com/gnome-design-team/gnome-mockups/tree/master/tweak-tool">design</a>.
The difference is stark; compare the old and the new versions below;</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/gnome/310.png"><img alt="New UI" src="http://www.johnstowers.co.nz/blog/static/images/gnome/310-sml.png" /></a></p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/gnome/304b.png"><img alt="Old UI" src="http://www.johnstowers.co.nz/blog/static/images/gnome/304b-sml.png" /></a></p>
<p>In addition to the use of new widgets (<code>Gtk.HeaderBar</code>, <code>Gtk.Application</code>,
<code>Gtk.Stack</code>, <code>Gtk.SearchBar</code>) the organisation of tweaks into categories
has been <a href="https://wiki.gnome.org/GnomeTweakTool">updated</a>. This should make many
settings easier to find, especially in conjunction with new translations for many
tweak names and descriptions.</p>
<p>Historically, the tweak tool UI was mostly auto-generated, resulting in a rather
uniform and boring look, and more importantly the inability to easily group tweaks
together to show causality (such as turning off desktop icons makes the options
to show specific types of icons on the desktop redundant). This architectural
limitation has now been fixed, and in addition, specialized UI elements have
been created for certain tweaks; startup applications, shell extensions,
desktop icons, the shell top bar, etc.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/gnome/310-desktop.png"><img alt="Desktop Icon Options" src="http://www.johnstowers.co.nz/blog/static/images/gnome/310-desktop-sml.png" /></a></p>
<p>Other highlights of 3.10 include;</p>
<ul>
<li>Allow updating GNOME Shell extensions from inside tweak tool</li>
<li>Startup application management</li>
<li>Offer to logout user when tweaks require the session restarted </li>
<li>GNOME style sidebar and search</li>
<li>Ability to disable middle-click paste (great for designers!)</li>
<li>Show text in tooltip when label is ellipsized, and make window maximizable and
resizable.</li>
<li>Better tweak names and descriptions (manage our own translations
instead of getting all from gsettings)</li>
</ul>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/gnome/310-startup.png"><img alt="Startup Applications" src="http://www.johnstowers.co.nz/blog/static/images/gnome/310-startup-sml.png" /></a></p>
<p>Unfortunately, not all features I wanted to implement were completed. Things I will
be working on in 3.12 include;</p>
<ul>
<li>Hidpi tweak (this will land in 3.10.1)</li>
<li>Better search interaction (focus stealing and search-results layout fixes)</li>
<li>Improved layout when the window is maximized</li>
<li>Resurrect the wacom panel (this was generously contributed at the start of the
cycle, but I had no time to port it to the new design, nor any way to test it)</li>
<li>Privileged helper for operations requiring root permissions (power management
options, installing system wide themes)</li>
<li>Keyboard layout specialized UI</li>
<li>Theme management UI</li>
</ul>
<p>For more information on GNOME 3.10 and GNOME Tweak Tool 3.10 check out
<a href="http://worldofgnome.org/tweak-tool-3-10-reveals-appmenus-in-applications/">world of gnome here</a>,
<a href="http://worldofgnome.org/csds-came-to-stay-in-gnome-3-10/">here</a>,
<a href="http://worldofgnome.org/gnome-upcoming-features/">and here again</a>, and
<a href="https://wiki.gnome.org/GnomeTweakTool">wiki.gnome.org</a>. Hope you all enjoy the
release.</p>Python Bindings to the Pointcloud Library2013-02-03T02:00:00+01:00John Stowerstag:www.johnstowers.co.nz/blog,2013-02-03:2013/02/03/python-pcl/<p>I'd like to announce the release of <a href="http://strawlab.github.com/python-pcl/">python-pcl</a>,
python bindings to the <a href="http://pointclouds.org">pointcloud</a> library.</p>
<p>This is not a full binding to the rather large PCL API. Currently, the following
parts of the API are wrapped</p>
<ul>
<li>I/O and integration; saving and loading PCD files</li>
<li>segmentation</li>
<li>sample consensus model fittting (RANSAC + others, cylinders, planes, common geometry)</li>
<li>smoothing</li>
<li>filtering</li>
</ul>
<p>The code tries to follow the Point Cloud API, and also provides helper function
for interacting with numpy.</p>
<p>A minimal example (demonstrating the <code>StatisticalOutlierFilter</code></p>
<div class="codehilite"><pre><span></span><span class="kn">import</span> <span class="nn">pcl</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">pcl</span><span class="o">.</span><span class="n">PointCloud</span><span class="p">()</span>
<span class="n">p</span><span class="o">.</span><span class="n">from_file</span><span class="p">(</span><span class="s2">"C/table_scene_lms400.pcd"</span><span class="p">)</span>
<span class="n">fil</span> <span class="o">=</span> <span class="n">p</span><span class="o">.</span><span class="n">make_statistical_outlier_filter</span><span class="p">()</span>
<span class="n">fil</span><span class="o">.</span><span class="n">set_mean_k</span> <span class="p">(</span><span class="mi">50</span><span class="p">)</span>
<span class="n">fil</span><span class="o">.</span><span class="n">set_std_dev_mul_thresh</span> <span class="p">(</span><span class="mf">1.0</span><span class="p">)</span>
<span class="n">fil</span><span class="o">.</span><span class="n">filter</span><span class="p">()</span><span class="o">.</span><span class="n">to_file</span><span class="p">(</span><span class="s2">"inliers.pcd"</span><span class="p">)</span>
</pre></div>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/strawlab/table_scene_lms400_both.png"><img alt="Filtering Example" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/table_scene_lms400_both_sml.png" /></a></p>
<p>The main limitation of the current implementation is that is only supports
the <code>PointXYZ</code> point type. PCL is a heavily optimized and templated API,
and the best method for creating specializations correspoinding to the correct point type
in a dynamic language like Python is not clear.</p>
<p>Nevertheless, the binding is already capable of smoothing, filtering and
the fitting of geometries in arbitary 3D point cloud data.</p>
<p>The binding is written using Cython, and is one of the more complex <code>C++</code>
bindings I could find. </p>
<p>The current release has been tested with </p>
<ul>
<li>pcl 1.5.1</li>
<li>Cython 0.16</li>
</ul>
<p>although it should work with more recent releases.</p>
<p>I would be interested in adressing the specialization issues using the recently
added and improved
<a href="http://docs.cython.org/src/userguide/fusedtypes.html">fused types support in Cython</a>.</p>
<p>This work has been supported by, and is currently in production use at,
<a href="http://strawlab.org">Strawlab</a>.</p>ROS and Gtk for Laboratory Control2013-01-27T23:15:32+01:00John Stowerstag:www.johnstowers.co.nz/blog,2013-01-27:2013/01/27/lab-with-gtk-ros-1/<p>At the lab in which I work <a href="http://strawlab.org">(Andrew Straw, strawlab)</a> we
study the visual flight behaviour of Drosophila using virtual reality. The
implementation of this will be explained in future posts and papers however for
this post I am going to describe how I used Gtk<sup id="fnref:1"><a class="footnote-ref" href="#fn:1" rel="footnote">1</a></sup> and
<a href="http://www.ros.org">ROS</a> to build an interface to control and monitor
running experiments (called the 'Operator Console').</p>
<p>A future post will address and release all the ROS+GObject<sup id="fnref:2"><a class="footnote-ref" href="#fn:2" rel="footnote">2</a></sup> glue that lets these
interfaces scale dynamically as nodes (dis)appear. This just shows the relevant
Gtk parts and has some comments on what I would like from Gtk to make these
sort of interfaces easier.</p>
<p>The screenshow shows the first tab of the 'Operator Console'<sup id="fnref:3"><a class="footnote-ref" href="#fn:3" rel="footnote">3</a></sup>.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/strawlab/oc-tab1.png"><img alt="Operator Console" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/oc-tab1-sml.png" /></a></p>
<p><strong><em> Implementation Notes </em></strong></p>
<ul>
<li>I use the secondary icon support of <code>Gtk.Entry</code> to show the contents contain
sensible data. Maybe validation support in Gtk would be useful here
<a href="https://bugzilla.gnome.org/show_bug.cgi?id=446056">bug</a>.</li>
<li>The 'Description' entry is a <code>Gtk.TextView</code>, not a <code>Gtk.Entry</code>. It was necessary
to apply custom CSS to make it look reasonably similar. Sadly, it does not
support the full/same set of CSS properties as <code>Gtk.Entry</code>, so it was impossible
to show the same border radius and focus colors
<a href="https://bugzilla.gnome.org/show_bug.cgi?id=687363">bug</a>. Perhaps a multi-line
<code>Gtk.Entry</code> would be better.</li>
<li>The bottom half of the window shows the utilisation of all computers. I tried
a few versions of this, and simple sensibly formatted monospaced text looked
much better than anything else I tried. Any suggestions?</li>
</ul>
<p>This screenshot shows an example screen where we mix the control and monitoring
or many instances of the same ROS node.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/strawlab/oc-tab2.png"><img alt="Operator Console" src="http://www.johnstowers.co.nz/blog/static/images/strawlab/oc-tab2-sml.png" /></a></p>
<p><strong><em> Implementation Notes </em></strong></p>
<ul>
<li>The <code>Gtk.Switch</code> simultainously displays the status of the projector, and also
allows control of the node. The is a common use-case in the software, and
due to the asynchronous nature of the ROS messages, I need to distinguish these
from user-generated signals. I have wrappers such as the following
for many widgets<sup id="fnref:4"><a class="footnote-ref" href="#fn:4" rel="footnote">4</a></sup>. Advice on how to distinguish this use-case would be
preferred.</li>
</ul>
<div class="codehilite"><pre><span></span><span class="k">class</span> <span class="nc">UpdateableGtkSwitch</span><span class="p">(</span><span class="n">Gtk</span><span class="o">.</span><span class="n">Switch</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">Gtk</span><span class="o">.</span><span class="n">Switch</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_changing</span> <span class="o">=</span> <span class="bp">False</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connect_after</span><span class="p">(</span><span class="s2">"notify::active"</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_changed</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_changed</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">):</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_changing</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">stop_emission</span><span class="p">(</span><span class="s2">"notify::active"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">set_active</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">is_active</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_changing</span> <span class="o">=</span> <span class="bp">True</span>
<span class="n">Gtk</span><span class="o">.</span><span class="n">Switch</span><span class="o">.</span><span class="n">set_active</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">is_active</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_changing</span> <span class="o">=</span> <span class="bp">False</span>
<span class="k">def</span> <span class="nf">connect</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connect_after</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
</pre></div>
<ul>
<li>The "Standby (Computer1)" is for display only and mirrors the status of a ROS
topic. I would like some way visually to inicate that this widget is not actually
an editable <code>Gtk.Entry</code>. Currently the <code>Gtk.Entry</code> is set
<code>editable = False</code>, it looks to out of place with <code>sensitive = False</code>.
Perhaps I should add some custom <code>CSS</code> to color it slightly different.
Suggestions are appreciated.</li>
</ul>
<h2>Closing Remarks</h2>
<p>I'm really happy with the status of the <code>PyGObject</code> bindings. We have a few quite
large applications built using them (and ROS) and I have no complaints about
performance<sup id="fnref:5"><a class="footnote-ref" href="#fn:5" rel="footnote">5</a></sup> or otherwise. The conventional wisdom was that PyGTK (and GTK) were
not suitable for threaded workloads but the threading model of ROS guarentees that
the 'operator-console' shown above manages upwards of 50 background threads asynchronously
updating the GUI state.</p>
<div class="footnote">
<hr />
<ol>
<li id="fn:1">
<p>Actually PyGObject, argh why didn't we keep the name as pygtk? <a class="footnote-backref" href="#fnref:1" rev="footnote" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:2">
<p>I'll blog about this later. For the curious,
<a href="https://github.com/strawlab/rosgobject">rosgobject</a> lives here. <a class="footnote-backref" href="#fnref:2" rev="footnote" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
<li id="fn:3">
<p>In real operation this GUI shows the state of many more
machines/nodes/computers. This screenshot is running on my laptop because
showing too much more might give away the game ;-). <a class="footnote-backref" href="#fnref:3" rev="footnote" title="Jump back to footnote 3 in the text">↩</a></p>
</li>
<li id="fn:4">
<p><code>freeze_notify</code> and <code>thaw_notify</code> would almost work, if the events
could be dropped and not queued. Also, not all widgets use <code>notify::active</code>,
<code>GtkComboBox(Text)</code> for example. A general way to do this would be preferred. <a class="footnote-backref" href="#fnref:4" rev="footnote" title="Jump back to footnote 4 in the text">↩</a></p>
</li>
<li id="fn:5">
<p>Excluding plotting / graphing performace. But that is fodder for a later post. <a class="footnote-backref" href="#fnref:5" rev="footnote" title="Jump back to footnote 5 in the text">↩</a></p>
</li>
</ol>
</div>I'm at GUADEC2012-07-27T02:38:26+02:00John Stowerstag:www.johnstowers.co.nz/blog,2012-07-27:2012/07/27/im-at-guadec/<p>An organised person would have blogged about going to GUADEC before the event started. I am not that person. I am there now. Come see me if you want to talk about / hack on</p>
<ul>
<li>
<p>GNOME tweak tool</p>
</li>
<li>
<p>Scientific computing with PyGObject (improving interaction with numpy, etc)</p>
</li>
<li>
<p>Windows builds of PyGObject + GTK3</p>
</li>
<li>
<p>Real time charting (<a href="https://github.com/nzjrs/uber-graph">https://github.com/nzjrs/uber-graph</a>)</p>
</li>
</ul>A Change2011-10-19T16:15:32+02:00John Stowerstag:www.johnstowers.co.nz/blog,2011-10-19:2011/10/19/a-change/<p>2011 has been an interesting year. Between the <a href="http://en.wikipedia.org/wiki/Christchurch_earthquake">stupid earthquakes</a> and the pressure of finishing my PhD, I have been silent because I have had nothing interesting to talk about (cf. twitter...).</p>
<p>But there is a light at the end, I'm on track to complete my thesis, <em>'Biologically Inspired Visual Control of Flying Robots',</em> in December/January.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/chch1.jpg"><img alt="Christchurch, demolished, the old..." src="http://www.johnstowers.co.nz/blog/static/images/imported/chch1-sml.jpg" /></a></p>
<p>I'm excited to say that I have accepted a job at the <a href="http://www.imp.ac.at">Institute of Molecular pathology</a>, in a <a href="http://strawlab.org/">research group</a> studying the mechanisms of visual flight control in insects. Technology wise, it is a perfect fit; the experimental apparatus involves a multi-camera real-time flight tracking system and estimator for multiple targets in an augmented reality flight arena. It is open-source (ish), and python/numpy. Research wise, it allows me to investigate some of the assumptions and unknowns in the biomimetic control systems I implemented during my PhD. And it is in Vienna, 1st Feb, 2012!</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/vienna1.jpg"><img alt="Vienna, the new..." src="http://www.johnstowers.co.nz/blog/static/images/imported/vienna1-sml.jpg" /></a></p>
<p>This is a career change for me. In the last few years it became increasingly clear that I was morally uncomfortable with the use of UAVs as weapons (drones). Previously I had consoled myself with there existing an ethical and philosophical difference between 'the application of research' and 'the action of research'. When It came to looking for work, and considering who to work for, this difference was often eroded.</p>
<p>It has also been particularly frustrating being in New Zealand for the last 12 months and watching our flaccid national response to the three recent challenges here (<a href="http://en.wikipedia.org/wiki/Rugby_World_Cup">world cup</a>, <a href="http://en.wikipedia.org/wiki/Christchurch_earthquake">earthquake</a>, <a href="http://en.wikipedia.org/wiki/2011_Tauranga_oil_spill">rena oil spill</a>).</p>
<p><strong>Technology Tidbits</strong></p>
<ul>
<li>
<p><a href="https://live.gnome.org/GnomeTweakTool">gnome-tweak-tool</a> is coming along nicely. I have some things in bugzilla to address, but I am happy with the state and direction.</p>
</li>
<li>
<p>GNOME 3 has helped my productivity greatly, maximized windows, keyboard navigation and less distractions have been welcome changes.</p>
</li>
<li>
<p><a href="http://www.johnstowers.co.nz/blog/index.php/2011/04/03/end-of-an-era-pygtk/">Pygtk is in maintenance mode</a> (I need it to keep working for some of <a href="http://waspuav.org/">my UAV code</a>). python-gobject and gobject-introspection are awesome and easy ways to write new GNOME apps.</p>
</li>
<li>
<p>LaTeX is awesome, and I am happy <a href="http://blogs.gnome.org/nacho/2011/10/02/gedit-latex-a-powerful-plugin-to-work-with-latex">we have returned</a> the <a href="https://live.gnome.org/Gedit/LaTeXPlugin">gedit-latex plugin</a> to being the best way to write LaTeX on Linux.</p>
</li>
<li>
<p><a href="http://www.phoronix.com/scan.php?page=news_item&px=MTAwMjY">Nothing of value is ever said in comments on the internet.</a></p>
</li>
</ul>
<p>This post has been brought to you by procrastination.</p>Interfacing Python + C + OpenCV via ctypes2011-07-15T12:01:48+02:00John Stowerstag:www.johnstowers.co.nz/blog,2011-07-15:2011/07/15/interfacing-python-c-opencv-via-ctypes/<p>I was recently asked to help a colleague access his image processing C-library from python; quite a common task. As those of you who are familiar with Python might realise, there are a whole bag of ways that this can be accomplished;</p>
<ul>
<li>
<p><a href="http://www.swig.org/">SWIG</a></p>
</li>
<li>
<p><a href="http://cython.org/">Cython</a></p>
</li>
<li>
<p>ctypes</p>
</li>
<li>
<p><a href="https://launchpad.net/pybindgen">PybindGen</a></p>
</li>
</ul>
<p>In this case the colleague only needed to access a single function from the library returning image data, and then hand this result onto OpenCV. One happy side effect of the new (> v2.1) python-opencv bindings is that they do <a href="http://opencv.willowgarage.com/documentation/python/cookbook.html#pil-image-to-opencv">no validation on CvImage.SetData</a>, which means you can pass an arbitrary string/pointer. Because of this I advised him I thought using something like SWIG was overkill, and he could just write a wrapper to his library using ctypes, or a thin python extension directly.</p>
<p>Image data contains embedded NULLs, and I could not find a concise example of dealing with non null-terminated, non-string char * arrays via ctypes so I wrote one.</p>
<div class="codehilite"><pre><span></span># char *test_get_data_nulls(int *len);
func = lib.test_get_data_nulls
func.restype = POINTER(c_char)
func.argtypes = [POINTER(c_int)]
l = c_int()
data = func(byref(l))
print data,l,data.contents
</pre></div>
<p>and, another approach</p>
<div class="codehilite"><pre><span></span># void test_get_data_nulls_out(char **data, int *len);
func_out = lib.test_get_data_nulls_out
func_out.argtypes = [POINTER(POINTER(c_char)), POINTER(c_int)]
func.restype = None
l2 = c_int()
data2 = POINTER(c_char)()
func_out(byref(data2), byref(l2))
print data2,l2,data2.contents
</pre></div>
<p>The full code can be found <a href="https://gist.github.com/990493">here</a> and contains examples showing how to deal with data of this type using ctypes, and by writing a simple python extension linking with the library in question.</p>End of an Era: PyGTK2011-04-03T14:48:08+02:00John Stowerstag:www.johnstowers.co.nz/blog,2011-04-03:2011/04/03/end-of-an-era-pygtk/<p>I just <a href="http://mail.gnome.org/archives/gnome-announce-list/2011-April/msg00002.html">released</a> PyGTK 2.24, which will almost certainly be the last major <a href="http://www.pygtk.org">PyGTK</a> release. The future of Python on the GNOME platform is <a href="http://live.gnome.org/PyGObject">PyGObject + GObject Introspection</a>. From my experience over the last few months porting a number of my projects, the future is bright.</p>
<p>In a cruel twist of irony, the state of PyGTK on Windows and Mac has never been better. The credit for the windows work (and some great documentation improvements this cycle) must go to Dieter Verfaillie</p>
<ul>
<li>
<p>Windows installer (<a href="https://github.com/dieterv/pygtk-installer">homepage</a>) (<a href="http://ftp.gnome.org/pub/GNOME/binaries/win32/pygtk/">download</a>)</p>
</li>
<li>
<p>Mac improvements (<a href="http://www.daa.com.au/pipermail/pygtk/2011-March/019520.html">all in one installer work</a>) (<a href="http://blog.xebia.com/2011/03/31/bundling-a-python-application-on-mac-os-x-with-virtualenv/">bundling advice</a>)</p>
</li>
</ul>
<p>I hope that the new stack will reach the same level of capability on other platforms as GTK+ 2.24, but in the large scheme of things the renewed development excitement surrounding GTK+ 3.0 and GNOME 3 is excellent consolation.</p>
<p>As a user I would like to thank those developers before me for creating PyGTK. It was the first 'pythonic' UI toolkit for linux, and a pleasure to use. As the recent maintainer of PyGTK I would especially like to thank those recent developers who helped me, in particular Dieter Verfaillie who really pushed PyGTK over the line regarding Windows support, into the great state it is now.</p>
<p>I'll leave you with some graphical statistics (<a href="https://gist.github.com/900087">generated</a> using <a href="http://scm-pepper.sourceforge.net/">pepper</a>) for the 12 year history of PyGTK. If planet strips the wordpress gallery then please <a href="http://www.johnstowers.co.nz/blog/index.php/2011/04/03/end-of-an-era-pygtk/">click here</a>.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/activity.png"><img alt="activity_m.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/activity_m.png" /></a></p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/activity-directories.png"><img alt="activity-directories_m.png " src="http://www.johnstowers.co.nz/blog/static/images/imported/activity-directories_m.png" /></a></p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/activity-auth.png"><img alt="activity-auth_m.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/activity-auth_m.png" /></a></p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/commit-counts.png"><img alt="commit-counts_m.png " src="http://www.johnstowers.co.nz/blog/static/images/imported/commit-counts_m.png" /></a></p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/loc.png"><img alt="loc_m.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/loc_m.png" /></a></p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/loc-tags.png"><img alt="loc-tags_m.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/loc-tags_m.png" /></a></p>
<p><strong>update:</strong> To clarify a point raised in the comments, <em>PyGTK will be maintained in the exact same was as the GTK+-2.0 series will be maintained</em>. Bug fix releases will be made if necessary, but no new features will be added. If you want the new GTK+-3.0 features then you should use <a href="http://live.gnome.org/PyGObject">PyGObject + GObject Introspection</a>.</p>
<p>The PyGTK code will not disappear from any servers, it will continue to be shipped in all distributions for the forseeable future, it will continue to work very well on windows, and many applications will continue to use it.</p>PyGTK All-in-one Installer for Windows2010-12-24T20:18:25+01:00John Stowerstag:www.johnstowers.co.nz/blog,2010-12-24:2010/12/24/pygtk-all-in-one-installer-for-windows/<p>The PyGTK team is pleased to <a href="http://www.daa.com.au/pipermail/pygtk/2010-December/019296.html">announce</a> the return of the highly popular all-in-one installer for Windows.</p>
<blockquote>
<p>The PyGTK All-in-one installer provides an alternative installation method for PyGTK users on Windows. It bundles PyGTK, PyGObject, PyCairo, PyGtkSourceView2, PyGooCanvas, PyRsvg, the gtk+-bundle and Glade in one handy installer.</p>
</blockquote>
<p>Currently 32 bit Python 2.6 and 2.7 versions are supported on Windows XP and above.</p>
<ul>
<li>
<p><a href="http://download.gnome.org/binaries/win32/pygtk/2.22/">Download</a></p>
</li>
<li>
<p><a href="https://github.com/dieterv/pygtk-installer/blob/master/README.rst">README</a></p>
</li>
</ul>
<p><a href="https://github.com/dieterv">Dieter Verfaillie</a> deserves enormous thanks for this work. Firstly, he performed the tedious job of ensuring that all the component MSI installers were exactly correct, and secondly, the really difficult task of deconstructing these individual installers and reassembling their contents into a single cohesive executable.</p>
<p>This is a true all-in-one installer, it does not simply call out to launch the individual MSI files.</p>
<p><img alt="" src="https://github.com/dieterv/pygtk-installer/wiki/screenshots/2.22.4/1-welcome.png" /><img alt="" src="https://github.com/dieterv/pygtk-installer/wiki/screenshots/2.22.4/2-customize.png" /><img alt="" src="https://github.com/dieterv/pygtk-installer/wiki/screenshots/2.22.4/6-changerepairmodify.png" /></p>
<ul>
<li>
<p>More screenshots <a href="https://github.com/dieterv/pygtk-installer/wiki">here</a>.</p>
</li>
<li>
<p>Please <a href="https://github.com/dieterv/pygtk-installer/issues">file bugs</a> as appropriate.</p>
</li>
<li>
<p>We are looking to collaborate with others who want to create gtk+ (and friends) all-in-one installers for Windows. We anticipate the tools to generate these installers will move to GNOME in future - perhaps in a common repository. Suggestions and feedback welcome.</p>
</li>
</ul>GTK+ Viewer for Microsoft Kinect2010-11-19T01:09:11+01:00John Stowerstag:www.johnstowers.co.nz/blog,2010-11-19:2010/11/19/gtk-viewer-for-microsoft-kinect/<p>I'm hacking on the new <a href="http://en.wikipedia.org/wiki/Kinect">Kinect</a> at the moment and the <a href="https://github.com/OpenKinect/libfreenect/blob/master/c/examples/glview.c">OpenGL viewer</a> didn't work for me so I threw together this <strong><a href="https://gist.github.com/704902">terrible</a></strong><a href="https://gist.github.com/704902"> quality gtk+ one.</a> I'll clean up the code and try to get this into <a href="http://openkinect.org/wiki/Main_Page">OpenKinect</a> ASAP.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/gtk-microsoft-kinect.jpg"><img alt="GTK+ viewer for Microsoft Kinect" src="http://www.johnstowers.co.nz/blog/static/images/imported/gtk-microsoft-kinect-sml.jpg" /></a></p>Map Updates2010-06-10T16:34:27+02:00John Stowerstag:www.johnstowers.co.nz/blog,2010-06-10:2010/06/10/map-updates/<p>I recently made a number of <a href="http://mail.gnome.org/archives/gnome-announce-list/2010-June/msg00003.html">improvements</a> to <a href="http://nzjrs.github.com/osm-gps-map/">osm-gps-map</a>, the easy to use mapping widget. The motivation for these came at the request of the <a href="http://www.foxtrotgps.org/">foxtrotGPS</a> developers (foxtrotGPS is a community developed fork of TangoGPS). These changes enhanced the API for adding <a href="http://github.com/nzjrs/osm-gps-map/blob/master/src/osm-gps-map-image.h">images</a> and <a href="http://github.com/nzjrs/osm-gps-map/blob/master/src/osm-gps-map-track.h">tracks</a> to the map, and in addition allowed me to clean up the <a href="http://github.com/nzjrs/osm-gps-map/blob/master/src/osm-gps-map-widget.h">basic API</a> making it easier to use for the common case. But, there is more, especially relevant to Gtk+/GNOME 3.0.</p>
<ul>
<li>
<p>Introspection Support (<a href="http://github.com/nzjrs/osm-gps-map/blob/master/examples/mapviewer.js">JavaScript example</a>)</p>
</li>
<li>
<p>GTK+ 3.0 Compatibility</p>
</li>
<li>
<p><a href="http://nzjrs.github.com/osm-gps-map/docs/reference/html/index.html">API Documentation</a> (finally...)</p>
</li>
<li>
<p>Improved examples (<a href="http://github.com/nzjrs/osm-gps-map/blob/master/examples/mapviewer.c">C</a>, <a href="http://github.com/nzjrs/osm-gps-map/blob/master/examples/mapviewer.py">Python</a>, <a href="http://github.com/nzjrs/osm-gps-map/blob/master/examples/mapviewer.js">JavaScript</a>)</p>
</li>
</ul>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/screenshot-lt-mapviewer-070.png"><img alt="osm-gps-map demo application" src="http://www.johnstowers.co.nz/blog/static/images/imported/screenshot-lt-mapviewer-070.png" /></a></p>Nautilus + Coverflow part two2010-02-05T02:21:19+01:00John Stowerstag:www.johnstowers.co.nz/blog,2010-02-05:2010/02/05/nautilus-coverflow-part-two/<p>After two complete rewrites following my initial experiments, the other <a href="http://gloobus.wordpress.com/">gloobus</a> developers <em>(badchoice and kitkat)</em> have <a href="http://gloobus.wordpress.com/2010/02/04/gloobus-flow/">continued to work</a> on integrating this coverflow view into nautilus.</p>
<p>The implementation of the coverflow widget this time is a little more sane. It is passed a GtkTreeModel of GFiles, and basically does everything in isolation. The coupling to nautilus is quite loose, so the idea is that the widget can be reused easily by others.</p>
<p>The code is available from the linked blog post, or <a href="http://github.com/nzjrs/nautilus">from github</a>.</p>Misc Hacking2010-01-22T11:22:39+01:00John Stowerstag:www.johnstowers.co.nz/blog,2010-01-22:2010/01/22/misc-hacking/<p>I had two days off while I moved offices, so I got a chance to catch up on my backlog of random hacking.</p>
<p><strong>osm-gps-map</strong></p>
<p>I released <a href="http://nzjrs.github.com/osm-gps-map/">osm-gps-map</a> v0.5.0 which adds a few new features (such as keyboard navigation) but also contains many bugfixes and performance improvements. Check the <a href="http://mail.gnome.org/archives/gnome-announce-list/2010-January/msg00063.html">release notes</a> for more information. The next item on the TODO is merging the OSD/layers branch.</p>
<p><strong>Conduit</strong></p>
<p>I <a href="http://mail.gnome.org/archives/gnome-announce-list/2010-January/msg00064.html">released Conduit 0.3.17</a> which was long overdue. Mostly a bugfix release and updating to new API. The <a href="http://live.gnome.org/Conduit">Conduit homepage</a> has also moved to live.gnome.org. Progress on Conduit is a bit slow at the moment, it does everything I want it to (I have a budget cellphone so phone synce does not interest me), and is pretty stable. I have some SOC work I would like to merge, but basically I am looking for developers and inspiration...</p>
<p><strong>PyGTK for Windows</strong></p>
<p>I finished off the fixes to build correct PyGTK+ installers on windows, hopefully closing <a href="https://bugzilla.gnome.org/show_bug.cgi?id=589671">bug #589671</a>. I uploaded new installers with the fixes people have reported. I expect these installers to become the 'final' installers at some point. Feedback welcome.</p>
<ul>
<li>
<p>Stable (requires the <a href="http://www.gtk.org/download-windows.html">Gtk+-2.16 bundle</a>)</p>
<ul>
<li>
<p><a href="http://www.johnstowers.co.nz/files/win32/pygobject-2.20.0.win32-py2.6.exe">PyGObject 2.20 for Windows</a></p>
</li>
<li>
<p><a href="http://www.johnstowers.co.nz/files/win32/pygtk-2.16.0.win32-py2.6.exe">PyGTK 2.16 (no Glade) for Windows</a></p>
</li>
<li>
<p><a href="http://www.johnstowers.co.nz/files/win32/pygtk-2.16.0+glade.win32-py2.6.exe">PyGTK 2.16 (with Glade) for Windows</a></p>
</li>
</ul>
</li>
<li>
<p><em>Update:</em> I also built</p>
<ul>
<li><a href="https://bugzilla.gnome.org/show_bug.cgi?id=589671#c34">PyCairo 1.8.6 for Windows</a></li>
</ul>
</li>
</ul>
<p><strong>PyGTK Hacking</strong></p>
<p>I wanted to play with the new client side windows work in Gtk+, so I <a href="http://gist.github.com/280520">ported the effects gtk-demo to Python</a>. This required a bit of ctypes magic to access the new API (good), and some more ctypes magic to interact with new signals that appears to have unfriendly prototypes (<a href="https://bugzilla.gnome.org/show_bug.cgi?id=607393">not so good, bug filed here</a>).</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/screenshot-mirrorpy.png"><img alt="PyGTK example using client side windows" src="http://www.johnstowers.co.nz/blog/static/images/imported/screenshot-mirrorpy.png" /></a></p>Gtk+ Map Widget2010-01-11T01:14:06+01:00John Stowerstag:www.johnstowers.co.nz/blog,2010-01-11:2010/01/11/gtk-map-widget/<p>It has been a long time between blogs. I thought I should talk about the piece of software that has been responsible for the most emails in my inbox over the last few days - <a href="http://nzjrs.github.com/osm-gps-map/">osm-gps-map, the Gtk+ based map widget</a>. What started as a widget for use in one small application of mine has grown considerably.</p>
<ul>
<li>
<p>I recently r<a href="http://mail.gnome.org/archives/gnome-announce-list/2010-January/msg00020.html">eleased 0.4.0</a>, a bugfix release.</p>
</li>
<li>
<p>I created a <a href="http://groups.google.com/group/osm-gps-map">mailing list</a>, if you are a user or interested osm-gps-map, them please join.</p>
</li>
</ul>
<p>But I thought I should take some time to highlight some of the most interesting users of osm-gps-map, particualry those users on the <a href="http://maemo.org/">Maemo</a> platform.</p>
<p><strong>Maep, OSM2Go and GPXView (<a href="http://harbaum.org/till/maemo/index.shtml">by Till Harbaum</a>)</strong></p>
<ul>
<li><a href="http://garage.maemo.org/projects/maep">Maep</a> seems to be the <a href="http://my-maemo.com/software/applications.php?name=Maep&fldAuto=1039&faq=38">flagship tile mapping application</a> for the Nokia n900. Most notable is the wonderful onscreen display that Till has created on top of osm-gps-map.</li>
</ul>
<p><a href="http://www.flickr.com/photos/nzjrs/4261506359/in/set-72157600285716789"><img alt="Maep Screenshot" src="http://farm3.static.flickr.com/2794/4261506359_b1cc82f94e_m.jpg" /></a> <a href="http://"><img alt="Maep" src="http://farm3.static.flickr.com/2760/4261505089_2039a7d8d4_m.jpg" /></a></p>
<ul>
<li><a href="http://harbaum.org/till/maemo/index.shtml">OSM2Go</a> is an editor for <a href="http://www.openstreetmap.org">openstreetmap.org</a> map data</li>
</ul>
<p><a href="http://www.flickr.com/photos/nzjrs/4262257708/in/set-72157600285716789"><img alt="OSM2Go" src="http://farm5.static.flickr.com/4001/4262257708_04fc93d1e0_m.jpg" /></a></p>
<ul>
<li><a href="http://harbaum.org/till/maemo/index.shtml">GPXView</a> is a viewer for <a href="http://www.geocaching.com">Geocaching.com</a> GPX files.</li>
</ul>
<p><a href="http://www.flickr.com/photos/nzjrs/4261505507/in/set-72157600285716789"><img alt="GPXView" src="http://farm3.static.flickr.com/2787/4261505507_a0a634999f_m.jpg" /></a></p>
<p><strong>BrainStorm (by <a href="http://www.aircrafter.org/boggs">Adam Boggs</a>)</strong></p>
<ul>
<li><a href="http://www.aircrafter.org/boggs/stormchasing/BrainStorm/">BrainStorm</a> is a storm chasing application that plots your track, and overlays current radar and severe warning imagery on the map.</li>
</ul>
<p><a href="http://www.flickr.com/photos/nzjrs/4262256864/in/set-72157600285716789"><img alt="Brainview" src="http://farm5.static.flickr.com/4012/4262256864_1ab2c2c578_m.jpg" /></a><a href="http://www.flickr.com/photos/nzjrs/4262255904/in/set-72157600285716789/"><img alt="" src="http://farm5.static.flickr.com/4007/4262255904_3e88b74927_m.jpg" /></a>
<strong>eCoach</strong></p>
<ul>
<li><a href="http://ecoach.garage.maemo.org/">eCoach</a> is an application for recording and managing sport activities with Nokia N900, it records heart rate data from various monitors, and plots your path on the map as you exercise.</li>
</ul>
<p><a href="http://www.flickr.com/photos/nzjrs/4261504055/in/set-72157600285716789/"><img alt="eCoach" src="http://farm5.static.flickr.com/4058/4261504055_fa389699be_m.jpg" /></a><a href="http://www.flickr.com/photos/nzjrs/4261588493/in/set-72157600285716789/"><img alt="" src="http://farm3.static.flickr.com/2711/4261588493_602e681ebd_m.jpg" /></a></p>
<p><strong>Conclusion</strong></p>
<ul>
<li>
<p>With the help of these users, the future of osm-gps-map looks very positive.</p>
</li>
<li>
<p>I am currently working on merging <a href="http://github.com/nzjrs/osm-gps-map/tree/master-till">Till's improvements</a> to master to discourage people from copying osm-gps-map source into their application.</p>
</li>
<li>
<p>If you are a user of osm-gps-map and I have forgotten you then I am sorry. Please <a href="http://www.johnstowers.co.nz/blog/index.php/about-me/">contact me</a> and join the osm-gps-map <a href="http://groups.google.com/group/osm-gps-map">mailing list</a>.</p>
</li>
</ul>Playing With Clutter2009-05-27T22:08:51+02:00John Stowerstag:www.johnstowers.co.nz/blog,2009-05-27:2009/05/27/playing-with-clutter/<p>Some of you out there might be familiar with <a href="https://launchpad.net/gloobus">Gloobus</a>. Over the last few nights I spent some time integrating Gloobus inside nautilus.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/nautilus-clutter.png"><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/nautilus-clutter-300x168.png" /></a></p>
<p>This is a proof of concept. I have done very little so far - it shows the first 8 files in the directory, and allows you to navigate between them with animation. It is mostly just a port of Gloobus from C++ into a ClutterGroup derived Actor in C, most of the thanks should go to the <a href="http://gloobus.wordpress.com/">Gloobus author</a>.</p>
<p>It features the same bugs as Gloobus, like poor support for resizing the window, positioning bugs, and it does not scale to very many files. It also leaks like a sieve (that one's on me). Here is my inadequacy represented in <a href="http://www.youtube.com/watch?v=1oZyFQM-m3E">video form</a>.</p>
<p>Those of you who regularly work with computers and other technology will be aware of how important it is to have a good quality internet connection. If you are looking into getting a new broadband connection then I recommend you look at <a href="http://www.o2.co.uk/broadband/">O2 uk broadband reviews</a>.</p>
<p>However, if you would still like to take a look and perhaps fix all the bugs, the steps for testing it are</p>
<ol>
<li>
<p>Install <a href="http://www.clutter-project.org/sources/clutter-gtk/0.9/">clutter-gtk-0.9</a></p>
</li>
<li>
<p>Download nautilus from my Git repository (the clutter branch)
<a href="https://github.com/nzjrs/nautilus/tree">https://github.com/nzjrs/nautilus/tree</a></p>
</li>
<li>
<p>Build the test program
cd src/file-manager
make -f Makefile.covflow && ./test-covflow</p>
</li>
<li>
<p>Built nautilus with --enable-clutter-view</p>
</li>
<li>
<p>Run the newly build nautilus
cd src
./nautilus -q
./nautilus --no-desktop</p>
</li>
</ol>
<p>This is very early work so the standard disclaimers apply. I needed to get this off my chest so I could get back to PhD work.</p>Map Widget Release2009-04-23T00:05:31+02:00John Stowerstag:www.johnstowers.co.nz/blog,2009-04-23:2009/04/23/map-widget-release/<p>I made a new April resolution to start blogging reguarly again. The first step of that long journey begins now.</p>
<p><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/screenshot-lt-openstreetmap-gps-map-1.png" /></p>
<p><strong>OpenStreetMap GPS Mapping Widget - 0.3</strong></p>
<p>I just released v0.3 of <a href="http://nzjrs.github.com/osm-gps-map/">osm-gps-map, the easy to use Gtk+ mapping widget</a>. Highlights for this release include;</p>
<ul>
<li>
<p>A new major contributor, <a href="http://github.com/mardy">Alberto Mardegan</a>, who worked on many of the new features of this release. Thanks a lot Alberto!</p>
</li>
<li>
<p>Draw map tracks with Cairo by default.</p>
</li>
<li>
<p>Interpolate between zoom levels while waiting for a tile to download.</p>
</li>
<li>
<p>Stop using GET_PRIVATE, and cache priv* for performance.</p>
</li>
<li>
<p>Keep an extra border of images offscreen for smoother scrolling at the edges of the map.</p>
</li>
<li>
<p>Keep the last N tiles in memory to improve render performance (previously they were loaded from disk)</p>
</li>
<li>
<p>Add some new api; osm_gps_set_center, osm_gps_map_scroll.</p>
</li>
</ul>
<p><strong>Conduit Hacking</strong></p>
<p>Hacking on Conduit continues, with much happyness for two reasons;</p>
<ul>
<li>
<p>GNOME has moved to git! Wahoo! I owe the sysadmin team many beers, except for Jc2k who I know could not handle them anyway.</p>
</li>
<li>
<p><a href="http://socghop.appspot.com/student_project/show/google/gsoc2009/gnome/t124022402214">Conduit got a SOC project</a>, congratulations <a href="http://blog.arosenfeld.net/">Alexandre</a>!</p>
</li>
<li>
<p>Work is ongoing to merge in the new configuration and settings serialization code.</p>
</li>
<li>
<p>More on this later, I promise...</p>
</li>
</ul>Some More Software2009-01-23T17:10:36+01:00John Stowerstag:www.johnstowers.co.nz/blog,2009-01-23:2009/01/23/some-more-software/<p>As was the theme of yesterdays post, here comes some more software that I have hacked on recently and can now be found on <a href="http://github.com/nzjrs/">Github</a>.</p>
<p><strong>osm-gps-map
</strong><a href="http://nzjrs.github.com/osm-gps-map/">osm-gps-map</a> is a Gtk+ widget (and Python bindings) that when given GPS co-ordinates, draws a GPS track, and points of interest on a moving map display. It Currently supports a number of different mapping sources;</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/openstreetmap-gps-map-small.png"><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/openstreetmap-gps-map-small.png" /></a></p>
<ul>
<li>
<p>openstreetmap (default)</p>
</li>
<li>
<p>openaerialmap</p>
</li>
<li>
<p>maps-for-free</p>
</li>
<li>
<p>satellite maps from a number of proprietary providers</p>
</li>
</ul>
<p>It also has the following features</p>
<ul>
<li>
<p>Intelligent caching of maps, including the ability to request a specific area of the map to be cached ahead of time</p>
</li>
<li>
<p>Recording of points of interest on the map (and the ability to add arbitary pixmaps at those points</p>
</li>
<li>
<p>Automatically draws a GPS track (a line showing the history of past added points)</p>
</li>
<li>
<p>Automatic centering on new GPS points</p>
</li>
<li>
<p>Support for multiple other tracks of co-ordinate points</p>
</li>
<li>
<p>Adjustable Zoom</p>
</li>
<li>
<p>Includes a <a href="http://github.com/nzjrs/osm-gps-map/blob/ef4c2f4e20fdb53c30f67356ffefed78e2c38aa6/src/main.c">comprehensive example</a></p>
</li>
<li>
<p>Simple, <a href="http://github.com/nzjrs/osm-gps-map/blob/ef4c2f4e20fdb53c30f67356ffefed78e2c38aa6/src/osm-gps-map.h">flat API</a></p>
</li>
</ul>
<p>I have been running it on a number of embedded boards (beagleboard, Overo) for some commercial work I have been doing (hence the minimal dependencies). For those Planet GNOME readers, It is kind of like <a href="http://blog.pierlux.com/projects/libchamplain/en/">libchamplain</a> but without the Clutter part.</p>
<p><strong>facebook-notify</strong>
Reaction to the <a href="http://www.johnstowers.co.nz/blog/index.php/2009/01/22/back-into-the-swing-of-things/">posting</a> of my <a href="http://nzjrs.github.com/facebook-notify/">Facebook notifier for GNOME</a> was positive, and I made a few quick fixes to remove some crasher bugs.</p>
<p>The biggest source of frustration at this point is that <em>libwebkitgtk segfaults, upon destroy, almost every time it encounters a page that requires the initialization of an NPAPI plugin</em>. <a href="http://github.com/nzjrs/facebook-notify/commit/4552e7cbe777af155396cb09d932352596e09dff">Disabling plugins</a> using the built in BrowserSettings object does not seem to have any effect.</p>
<p>Is there a RoadMap or any plans for an updated WebKitGtk release before the next round of distro updates?</p>
<p><strong>Conduit</strong>
I have been hacking on Conduit a bit over the last few days, and thanks to Julien Lavergne we now have updated builds for Intrepid and Jaunty. Julien also packaged a trunk snapshot (and set up the infrastructure to allow easier updating of such snapshots). I recommend all users of Conduit <a href="https://launchpad.net/~conduit/+archive">add our PPA</a>.</p>Back into the swing of things2009-01-22T01:38:06+01:00John Stowerstag:www.johnstowers.co.nz/blog,2009-01-22:2009/01/22/back-into-the-swing-of-things/<p>It has been a long time between posts, but I am back in New Zealand now, and vowing to get back on top of this blogging thing. I will post more in the coming weeks about my time in France with the <a href="http://www.enac.fr/">ENAC</a> crew, but I will start today small; by releasing some code I have been hacking on.</p>
<p>I wanted a small project to get back into the desktop development mindset after being in the embedded C (ARM 7 LPC2148) and OpenEmbedded (Beagleboard and Gumsix Overo) world for a while. I am also ecstatic that git was clearly the most popular DVCS in the recent survey, I really hope it provides the impetus for GNOME to move to a DVCS.</p>
<p>With that in mind I decided to use this small project to learn git. Firstly, <a href="http://github.com/nzjrs">github.com</a> is the coolest thing I have ever seen. I am moving all of my <a href="http://www.johnstowers.co.nz/blog/index.php/projects/">projects</a> (except <a href="http://www.conduit-project.org">Conduit</a>) there, and using the pages and wiki features to document them as appropriate.</p>
<p><strong>Facebook Notify</strong></p>
<p><a href="http://nzjrs.github.com/facebook-notify/">Facebook Notify</a> is a small PyGtk application which monitors your Facebook profile and notifies you when it changes. Some would consider it a gross abuse of the GNOME notification area, but it keeps me out of the web browser, decreasing procrastination, thereby improving my productivity.</p>
<p><img alt="" src="http://nzjrs.github.com/facebook-notify/screenshot3.png" />
<img alt="" src="http://nzjrs.github.com/facebook-notify/screenshot1.png" /></p>
<p>It notifies you when any of the following events occur;</p>
<ul>
<li>
<p>One of your friends changes their status, profile picture, or profile details</p>
</li>
<li>
<p>You receive a friend request, event or group invitation</p>
</li>
<li>
<p>Someone writes on one of your friends walls</p>
</li>
<li>
<p>One of your friends is tagged in a photo</p>
</li>
</ul>
<p>The code is a few days work, and nothing impressive. It does however contain a rather nifty deferred threaded network io system so to never block the GUI. I guess something similar could be done in C + libsoup + <a href="http://docs.dronelabs.com/gtask/">GTask</a>, if one was allowed to arbitarily chain callback functions. I remeber seeing a bug and discussion about this somewhere.</p>
<p>The main bug is that <em>WebKitGtk segfaults if you destroy the WebView after logging into Facebook</em>. Not cool. Anyone know anything about this bug? Is anyone going to make an updated <em>libwebkitgtk</em> release soon?</p>
<p>So, please, fork my code. That way I have a real life example to practice merging from!</p>iPhone Synchronization on Linux2008-10-16T22:28:57+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-10-16:2008/10/16/iphone-synchronization-on-linux/<p>Now that Matt has gone ahead and <a href="http://matt.colyer.name/2008/10/08/iphone-linux-syncing-with-conduit/">announced this</a>, I think I should send some more traffic his way. If you are interested in using Conduit to synchronize your iPhone contacts, calendar and notes then go and <a href="http://matt.colyer.name/projects/iphone-linux/index.php?title=Conduit_Syncing">check out his work</a>.</p>
<p><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/iphone-sml.jpg" />
<em>I hear people like this "iPhone" thingee</em><a href="http://www.johnstowers.co.nz/blog/static/images/imported/iphone.jpg">
</a></p>
<p>Also, I am still looking for volunteers to help me maintain the Google contacts/calendar Conduit dataproviders. They need some love, and I am just one developer.</p>One Month In France2008-10-05T11:29:04+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-10-05:2008/10/05/one-month-in-france/<p><strong>ENAC</strong></p>
<p>Hi Everyone, Its been a long time between blogging but I have an excuse. I have moved from <a href="http://maps.google.com/?ie=UTF8&ll=-43.526724,172.656026&spn=0.006628,0.021973&t=h&z=16">Christchurch New Zealand</a>, to <a href="http://www.enac.fr/">ENAC</a>, <a href="http://maps.google.com/maps?f=q&hl=en&geocode=&q=enac,+toulouse,+france&ie=UTF8&ll=43.565327,1.474915&spn=0.006623,0.021973&t=h&z=16">Toulouse, France</a>. I have now been here for a month, working with the <a href="http://paparazzi.enac.fr/wiki/index.php/Main_Page">UAV team</a> here.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/cimg3107.jpg"><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/cimg3107.jpg" />
</a><em>Screen Envy?</em><a href="http://www.johnstowers.co.nz/blog/static/images/imported/cimg3107.jpg"></a></p>
<p>The work has been really challenging, and I have settled into my routine, working towards some things I would like to demonstrate before I leave. I have spent a few weeks doing a lot of electronics design, updating the paparazzi autopilot board, the IMU, and the GPS boards. Nothing revolutionary, just some evolutionary improvements over the previous hardware.</p>
<ul>
<li>
<p>Consolidation of the interfaces between the main board, and the IMU+GPS+Radio+Motors. IMU interface is now SPI only, GPS interface is I2C only / UART only.</p>
</li>
<li>
<p>Addition of a 24bit ADC on the main board to directly measure the pressure sensor, no more op-amp+calibrate the offset at startup.</p>
</li>
<li>
<p>Physically smaller stackable board design.</p>
</li>
</ul>
<p>A lot of this work has been done with an eye towards moving some of the off-board vision processing I am currently doing onto the flying aircraft. I have been experimenting with the <a href="http://beagleboard.org/">beagleboard</a>, and one of the goals of the hardware refactoring above is to free up an interface to push data between the beagleboard and the flight controller. Probably I2C or UART, I am not sure yet.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/overo.jpg"><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/overo.jpg" />
</a><em>Can anyone get hold of a <a href="http://www.gumstix.net/Overo/cat/Overo/115.html">Gumstix Overo</a> for me?</em><a href="http://www.johnstowers.co.nz/blog/static/images/imported/overo.jpg"></a></p>
<p>I am hoping to be able to demonstrate some biomimetic control responses from my onboard vision system, using image motion information. I also hope to demonstrate hybrid external position estimation system using an off aircraft 3d vision system aided (kalman estimator) by on-board IMU .</p>
<p>Plenty of work for me ahead.</p>
<p><strong>Ubuntu</strong></p>
<p>I finally managed to upgrade to the Ubuntu Intrepid beta. I was pleased to see that it contained all sorts of productivity improvements;</p>
<ul>
<li>
<p>I used to waste about an hour a day keeping up with the US election news on Youtube, watching Sarah Palin insult the intelligence of all mammals on the planet with her existence. Intrepid fixed this for me by removing the feature where sound embedded in flash videos was played through the soundcard of my computer. Phew, thats a relief. I guess I will just need to go and watch Fargo instead.</p>
</li>
<li>
<p>Keeping in contact with my family via Skype was also a PITA, luckily Intrepid removed the ability for me to do that too, no sound to hear my parents nag me, and no video which would let them see me all hung over and tired.</p>
</li>
</ul>
<p>Im sure everyone reading this is aware of that feeling when you go and use a friends brand new $2000 Windows Vista computer. The way it runs so slowly with 2GHz of processing power at its disposal, crashes all the time and takes 6 minutes to turn on. It is brand new FFS. When I am in that situation it makes me feel like the entire engineering profession has failed me.</p>
<p>I got that feeling with Ubuntu this week.</p>
<p><strong>Conduit</strong></p>
<p>Unfortunately I have not been able to work on Conduit very much over the last month, and it appears that no one else seems to have had the time to either. This upgrade pain has destroyed my motivation, and I only just recovered from the previous month, where approximately 14,000 people reminded me that the Conduit GUI made them vomit in their mouth. Some positive re-inforcement (and some help hacking) would be a welcome change about now.</p>Blog as Noticeboard2008-08-23T01:45:53+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-08-23:2008/08/23/blog-as-noticeboard/<p><strong>Summer of Code</strong>
This has now finished, and I am really happy with how it went. I was able to complete a Python binding to libsyncml. This was done with the help of <a href="https://launchpad.net/pybindgen">Pybindgen</a>, which aside from a few quirks, performed admirably. Expect this to become the premier tool to automatically create python bindings to C/C++ libraries. The binding still contains too many bugs to be considered usable in Conduit trunk (read: crasher bugs) but I can see the light at the end of the tunnel.</p>
<p>Alexandre Rosenfeld was also successful in his <a href="https://code.launchpad.net/~airmind/conduit/gsoc08">Conduit Summer of Code project.</a> He contributed audio and video support for the iPod, and a comprehensive audio/video converter/trans-coder using python-gstreamer. The iPod support seems quite comprehensive, and the converter component is a necessary component of the Conduit architecture for our future plans.</p>
<p><img alt="iPod" src="http://www.johnstowers.co.nz/blog/static/images/imported/dbd_logo_final.jpg" /></p>
<p><strong>Conduit</strong>
While it may look like I have been dormant at Conduit hacking, work has been ongoing in a number of branches. <em><rant>Unfortunately I still cannot push these branches to bzr-playground.gnome.org because the SSH keys from GNOME have not been synced across in over two weeks, leaving me locked out.</rant></em>. The GIO port is now working (with the exception of gvfs <a href="http://bugzilla.gnome.org/show_bug.cgi?id=547133">bug #547133</a>, which I would dearly like someone to commit the fix for).</p>
<p>One of the major tasks necessary for the GIO port was the isolation of the platform specific parts, such as GConf, and GnomeVFS. One thing that fell out of this work is that Conduit now works on Windows. With no (~10 lines) code changes. Amazing really. It should be noted that this is not actually using GIO on windows, it is using a pure Python File class implementation.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-windows.png"><img alt="Conduit running on Windows" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-windows-small.png" /></a>
<em>Conduit can haz Windows!</em></p>
<p>I am not really serious about maintaining this port, but it shows what is possible. If someone wants to hack on this I can point them to the necessary places. But basically you will need</p>
<ul>
<li>
<p><a href="http://www.python.org/download/releases/2.5/">Python 2.5</a></p>
</li>
<li>
<p>The <a href="http://gladewin32.sourceforge.net/">Gtk+ Runtime for Windows</a></p>
</li>
<li>
<p><a href="http://ftp.gnome.org/pub/GNOME/binaries/win32/pycairo/1.4/">PyCairo</a>, <a href="http://ftp.gnome.org/pub/GNOME/binaries/win32/pygobject/2.14/">PyGObject</a>, <a href="http://ftp.gnome.org/pub/GNOME/binaries/win32/pygtk/2.12/">PyGtk</a> installers</p>
</li>
<li>
<p><a href="http://ftp.gnome.org/pub/GNOME/binaries/win32/goocanvas/0.10/">Goocanvas</a> and <a href="http://files.conduit-project.org/releases/pygoocanvas-0.10.0.win32-py2.5.exe">PyGoocanvas</a> installers</p>
</li>
<li>
<p>The conduit-win32 branch (not accessible ATM, due to aforementioned bzr brokeness)</p>
</li>
</ul>
<p>I have also moved over to using <a href="http://code.google.com/p/pywebkitgtk/">PyWebkitGtk</a> for the Conduit Web browser. They just made a 1.0 release, and I would really like it if those responsible for packaging Conduit, could please also package PyWebkitGtk, and ensure that it gets the necessary exceptions so that it is accepted into the appropriate distributions. Words cannot express how happy I am to be rid of gtkmozembed. It is a shame that webkitgtk was not accepted as an external dependency for GNOME 2.24, as this now makes getting things like pywebkitgtk into distributions a royal PITA.</p>
<p><strong>Openstreetmap GPS Widget</strong>
<a href="http://www.johnstowers.co.nz/blog/index.php/2008/05/21/frantic/">Some time ago</a> I mentioned the osm-gps-map widget that I have been working on, semi-related to my PhD. I just made the inaugural 0.1 release. This widget basically lets one easily write moving-map display widgets very easily, showing points of interest, and multiple gps paths atop of tiles fetched from <a href="http://openstreetmap.org/">openstreetmap</a>, or other mapping providers. It started as a port of <a href="http://www.tangogps.org/gps/cat/News">Tango GPS</a>, and can now basically do everything that application did, but behind a simple GObject API. Python bindings are also provided</p>
<p><img alt="Openstreetmap GPS Gtk Widget" src="http://www.johnstowers.co.nz/blog/static/images/imported/osm-gps-map.png" /></p>
<ul>
<li>
<p><a href="http://nzjrs.github.com/osm-gps-map">Download</a></p>
</li>
<li>
<p>Documentation (soon)</p>
</li>
<li>
<p><a href="http://nzjrs.github.com/osm-gps-map">Example Code</a></p>
</li>
<li>
<p>Applications <a href="http://www.flickr.com/photos/nzjrs/2786524612/">[1]</a> <a href="http://www.flickr.com/photos/nzjrs/2785669843/">[2]</a> using this widget.</p>
</li>
</ul>
<p><strong>Miscellany</strong></p>
<ul>
<li>
<p>Dear Ubuntu. Please make<a href="http://willwill100.deviantart.com/art/Interpid-Ibex-Mockup-Part-2-93584910"> this your theme</a>.</p>
</li>
<li>
<p><a href="http://homepages.ihug.co.nz/~corwin/rms08.html">RMS spoke at the University of Canterbury last week</a>. It was quite good and well attended.</p>
</li>
<li>
<p>I am off to France in 10 days for a two month PhD excursion/exchange to <a href="http://www.enac.fr/">ENAC</a></p>
</li>
<li>
<p>If you know a packager for a distribution, then poke them to package PyWebkitGtk</p>
</li>
</ul>A Frustrating Week2008-08-10T22:05:43+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-08-10:2008/08/10/a-frustrating-week/<p>It all started so well. While Conduit was not accepted into GNOME 2.24, it was <a href="http://mail.gnome.org/archives/devel-announce-list/2008-August/msg00001.html">blessed as an external dependency</a> for this cycle. That is great news for online service integration, and synchronization on the GNOME desktop. Congratulations to everyone who has helped me work on Conduit over these years, and well done to those who made it into the release set.</p>
<p>I was also able to make a <a href="http://ftp.acc.umu.se/pub/GNOME/sources/conduit/0.3/conduit-0.3.13.news">0.3.13</a> release incorporating those fixes and UI improvements I discussed last time around. After that things started to get worse.</p>
<p><img alt="Conduit 0.3.13.1 (and a hidden new feature)" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-03131-small.png" /></p>
<ol>
<li>
<p>The disk holding my /home partition crashed. Taking a whole bunch of stuff with it. Like a good geek most of it was backed up or in a RCS, with the exception of my SSH key. FAIL.</p>
</li>
<li>
<p>This meant I was unable to make the now customary <a href="http://ftp.acc.umu.se/pub/GNOME/sources/conduit/0.3/conduit-0.3.13.1.news">Conduit x.x.x.1 brown paper bag release</a> in time for the GNOME release. I do owe many thanks to our <a href="http://blogs.gnome.org/ovitters">amazing sysadmin, Olav,</a> for getting my new key onto the GNOME servers in a matter of hours.</p>
</li>
<li>
<p>The disk crash took with it a week+ of my SOC work, making me very angry. To calm the tempers I thought I would work on the conduit-gio branch.</p>
<ol>
<li>Unfortunately I am not certain if this has much chance of success. I seem to have hit a regression/gvfs <a href="http://bugzilla.gnome.org/show_bug.cgi?id=547133">bug #547133</a>.</li>
</ol>
</li>
<li>
<p>Then Christian Schlotter contributed a really nice Conduit dataprovider which supports sync/backup to Amazon S3.</p>
<ol>
<li>But this too seems blocked by a gvfs <a href="http://bugzilla.gnome.org/show_bug.cgi?id=547020">bug #547020.</a></li>
</ol>
</li>
<li>
<p>I then realized the implications of Webkit not being blessed as an external dependency. It means that I have two choices. Continue to drag around the pain that pthon-gtkmozembed brings, or move to using the unreleased (Py)WebkitGtk. I really encourage the WebkitGtk and PyWebkitGtk teams to make stable releases ASAP, to ensure they get into $DISTRO.</p>
</li>
<li>
<p>Then I found that I could not push to bzr-playground.gnome.org anymore. It appears that it did not pick up my new SSH key. And loggerhead source code browsing is broken there.</p>
</li>
<li>
<p>Completely fed up with software, I thought I would play with my new hardware toy. This is destined to replace the terrible, unsupported mess that was the <a href="http://www.phytec.com/products/sbc/ARM-XScale/phyCORE-ARM9-LPC3180.html">Phytec phyCORE LPC3180</a> module for the purposes of my PhD work.</p>
</li>
</ol>
<p><img alt="Beagleboard" src="http://www.johnstowers.co.nz/blog/static/images/imported/2420765852_063c30160b.jpg" />
<em>My brand new <a href="http://beagleboard.org/">beagleboard</a> (<a href="http://www.flickr.com/photos/koenkooi/2420765852/in/pool-beagleboard">photo</a>)</em></p>
<p>But then the FAIL continued. <a href="http://www.openembedded.org">http://www.openembedded.org</a> has been down for the last 5 days, making it impossible for me to check out the <a href="http://elinux.org/BeagleBoardAndOpenEmbeddedGit">openembedded tree that supports this board</a>. Does anyone know why it is down?</p>Conduit UI Experiments - Some Conclusions2008-08-04T02:49:03+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-08-04:2008/08/04/conduit-ui-experiments-some-conclusions/<p>Thanks once again to the many people who commented on my last two <a href="http://www.johnstowers.co.nz/blog/index.php/2008/07/31/conduit-ui-experiments/">blog</a> <a href="http://www.johnstowers.co.nz/blog/index.php/2008/08/01/conduit-ui-experiments-part-two/">posts</a>. All the feedback has been gratefully received. I just released <a href="http://www.conduit-project.org/wiki/0.3.13">Conduit 0.3.13</a> which, amongst other things, features some UI improvements based upon this feedback.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-examples.png"><img alt="Conduit with an example shown" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-examples-small.png" /></a><a href="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-after-example.png"><img alt="Conduit after some examples have been added" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-after-example-small.png" /></a><a href="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-0313-some-dps.png"><img alt="Conduit 0.3.13 showing multiple dataproviders" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-0313-some-dps-small.png" /></a>
<em>Improvements to the Conduit user interface.</em></p>
<p>Technically speaking, the release adds a knowledge framework to the application. This makes it easy to assist the user as they work through the interface. New features, utilizing this framework are;</p>
<ul>
<li>
<p>Conduit now ships with a few example synchronization groups. These groups are shown to the user based upon the devices connected, and the data providers loaded. When the user selects an example, it is added to the canvas automatically.</p>
</li>
<li>
<p>I have added a message area to the bottom of the window. This provides a limited number of hints to the user depending on their interactions with the application.</p>
</li>
</ul>
<p><strong>Coming Up</strong>
This release also lay a lot of the foundations for the few remaining big ticket items on the 0.4.0 TODO list. In the very near future we will see</p>
<ul>
<li>
<p>Merging GIO port</p>
</li>
<li>
<p>Merging of Alexandre's SOC work, which will improve sound and video support in Conduit</p>
</li>
<li>
<p>The first release supporting Windows Mobile synchronization</p>
</li>
<li>
<p>Miscellaneous PIM improvements (Improved Google support, ability to merge attributes during conflict)</p>
</li>
</ul>
<p><a href="http://blogs.gnome.org/johncarr">John Carr</a> is currently working on the ATK accessibility implementation. As far as we can tell, there are no other PyGTK applications that have implemented accessibility in manner we wish to (manual implementation of the Atk interfaces from a objects not derived from GtkWidget). Lucly he works with <a href="http://mark.doffman.com/">Mark</a>!</p>
<p>As far as using and distributing Conduit goes, it may get painful before it gets better. The next release will likely depend on PyGObject > 2.17 (for GIO support), and maybe <a href="http://live.gnome.org/PyWebKitGtk">PyWebkitGtk</a> (SVN because of lack of recent, stable, compatible releases of this and WebkitGtk). My experience with (py)GIO so far has been exceptionally positive, and those out there who have used the train wreck that is gtkmozembed before will appreciate my desire to move to PyWebkitGtk.</p>Conduit UI Experiments - Part Two2008-08-01T12:57:03+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-08-01:2008/08/01/conduit-ui-experiments-part-two/<p>Thank you everyone for their <a href="http://www.johnstowers.co.nz/blog/index.php/2008/07/31/conduit-ui-experiments/#comments">constructive comments</a> on my <a href="http://www.johnstowers.co.nz/blog/index.php/2008/07/31/conduit-ui-experiments/">previous post</a>. A number of posters suggested the rounded boxes that conduit uses to group dataproviders were too bold, unnecessary, or both. I would like your opinion on which of the following four choices is the most suitable as a replacement.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-boxes.png"><img alt="Four different styles showing how to differentiate between dataproviders" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-boxes-small.png" /></a>
<em>Which is the best; 1,2,3 or 4?</em></p>
<p>Number four is the current implementation. I am leaning towards the first option as a replacement. From testing it seems to look the most consistent on different themes, and I believe it is the most suitable background to place (future) drag and drop hints on.</p>Conduit UI Experiments2008-07-31T23:55:47+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-07-31:2008/07/31/conduit-ui-experiments/<p>The <a href="http://mail.gnome.org/archives/desktop-devel-list/2008-July/msg00202.html">discussions regarding my proposal</a> for inclusion of Conduit into GNOME 2.24 seem to be going OK. The inevitable issue of Conduit's user interface has been raised, and it is good to see some constructive comments being posted.</p>
<p>It is a bit of a simplification, but I see basically three schools of thought with regard to the user interface for synchronizing things. What follows is a brief discussion of the pros and cons of each approach.</p>
<ol>
<li>
<p>Everything should be automatically configured, and the current Conduit user interface should cease to exist - synchronization functionality/UI should be put into the respective application, calling into Conduit over DBus.</p>
<ol>
<li>
<p>While I agree in theory, there is a enormous divide between theory and practice. For the foreseeable future it is not going to be possible to auto-configure synchronization parameters for all mobile devices. This is in part due to quirks in the devices, and in part because its sometimes not possible to enumerate all the things that a device supports without first configuring/pairing/connecting/etc with it.</p>
</li>
<li>
<p>Many web services require a authorization step, either once per application, or once per session. To make this easier on the user, Conduit hosts its own web browser for showing the login dialog in a consistent manner. I believe that this is the least surprising result to the user, if they sync something in conduit, the authentication should be done in conduit, not in the users default web browser, where they may not immediately recognize that it is Conduit that requires the authentication. Until we get some more online-desktopy stuff into GNOME (sharing auth cookies, authenticating services, etc) I think our approach is a good one.</p>
</li>
<li>
<p>I think it is important to have a single, consistent interface for resolving conflicts, this is currently in the Conduit GUI, but perhaps we could export it over DBus, like we do for the configuration dialogs.</p>
</li>
<li>
<p>Doing everything in other applications, reduces Conduits power to the lowest common denominator of these applications. Some of the cool power of Conduit is in the synchronization partnerships you can create that are not even tied to an application. For example, its possible to configure a 'sync' from Youtube, onto your iPod, that downloads and transcodes videos from you favorite user/channel. In what app would something like that go?</p>
</li>
</ol>
</li>
<li>
<p>The conduit GUI should be simpler once you have configured everything.</p>
<ol>
<li>
<p>This is the approach I agree with the most. I think. This could potentially involve some sort of minimize/maximize type button that when pushed, would collaps the main canvas down into something like the iSync user interface.</p>
</li>
<li>
<p>While I quite like that approach, I would almost rather implement an entire second GUI using only the Conduit DBus interface. This is the approach being persued by Alexandre (SOC student working on iPod support), and by John Carr (for a simplified windows mobile experience)</p>
</li>
<li>
<p>It was also suggested to make configuration easier, we should try gtk.Assistant.</p>
</li>
</ol>
</li>
<li>
<p>Redesign the entire UI</p>
</li>
</ol>
<p><strong>Redesigning the Conduit GUI</strong>
Today I decided to work on improving the current UI based upon some feedback on <a href="http://mail.gnome.org/archives/desktop-devel-list/">desktop-devel-list</a>. First I spoke with my user interaction designer friend, and asked her what she thought. After 15 minutes of talking to an imaginary person that did not exist (any volunteers to be my interaction designer friend?) I decided to instead make some incremental improvements to the current UI. Some people may see this as <em>re-arranging deck chairs on the Titanic</em>, but hey, what have I got to lose?</p>
<p>The first step was to use the current theme colors. After a bit of looking around, I couldn't actually find a simple way to see what these were, so I wrote a quick application that draws color swatches for all the theme colors, in each of the widget states. If anyone wants so play with it, the code is available from my <a href="http://gist.github.com/49799">git repository</a>.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/theme-swatch.png"><img alt="Theme color swatches" src="http://www.johnstowers.co.nz/blog/static/images/imported/theme-swatch-small.png" /></a></p>
<p><em>Rounded rectangles make anything pretty.</em></p>
<p>With an idea of what colors are actually present, I changed the Conduit and DataProvider colors to use sensible colors from the theme. I also implemented some subtle gradients to make things a bit more GNOME.</p>
<p><img alt="Gross gradient colors. Just joking of course." src="http://www.johnstowers.co.nz/blog/static/images/imported/yuck-gradient.png" />
<em>Just kidding.</em></p>
<p>The result turned out pretty nice I think. This is the first time I have actually tried to make a custom widget match the user theme, and it was not too painful. One annoying limitation is that I only seem to get access to the matching style, but I would quite like to use colors from the tooltip style. It also gave me the opportunity to remove some UI clutter; I no longer show sources and sinks in different colors. Anyway the final result was;</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-themed.png"><img alt="Conduit matching the system theme" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-themed-small.png" /></a>
<em>Beautiful in brown, blue and black.</em></p>
<p>When I said subtle I really meant it. I would really appreciate it if a talented artistic hacker had a look at the code, and experimented with some more creative gradient settings. The function in question is <em>get_style_properties</em> which lives in <em>conduit/gtkui/Canvas.py</em>, and is implemented in all <em>CanvasItem</em> derived classes.</p>
<p>I then moved onto experimenting with waking the user through the sync and configuration process. I did some experiments using the Gedit message area to suggest steps the user could take. Thanks to Colin and <a href="http://www.gnome.org/projects/hotssh/">HotSSH</a> for the widget. Im not sure about this one.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/msgarea.png"><img alt="Conduit using a message area to make suggestions to the user" src="http://www.johnstowers.co.nz/blog/static/images/imported/msgarea-small.png" /></a>
<em>Too much like <a href="http://en.wikipedia.org/wiki/Office_Assistant">Clippy</a>?</em></p>
<p><a href="http://mail.gnome.org/archives/desktop-devel-list/2008-July/msg00299.html">Mathias suggested</a> a visible drop target. I think this will be my next challenge. I also did an example implementation showing what the Conduit UI would look like if the outer, grouping, rounded rectangle was removed. Now a simple line differentiates different between conduits. What do you think?</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-no-box.png"><img alt="Conduit with the outer rectangle removed, a single line divides conduits." src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-no-box-small.png" /></a>
<em>Now with 29% less rounded rectangles.</em></p>
<p><strong>Summary</strong></p>
<ul>
<li>
<p>I'm pretty happy with the themed colors. The work is available in my <a href="http://bzr-playground.gnome.org/~jstowers/conduit/devel/">bzr branch</a>, and will make it to trunk soon</p>
<ul>
<li><em>bzr branch http://bzr-playground.gnome.org/~jstowers/conduit/devel</em></li>
</ul>
</li>
<li>
<p>There is continued discussion regarding UI ideas happening on <a href="http://live.gnome.org/JohnCarr/Conduit">live.gnome.org</a></p>
</li>
<li>
<p>My Gtk theme swatch too is available from</p>
<ul>
<li><em>git clone http://gist.github.com/49799</em></li>
</ul>
</li>
<li>
<p>There will be a Conduit UI, of some sort, for the <a href="http://uk.youtube.com/watch?v=mvrva8NoMLM">forseeable future</a>. Lets spend some time making it better!</p>
</li>
</ul>
<p><strong>Updates</strong></p>
<ul>
<li>
<p>Fixed BZR Urls.</p>
</li>
<li>
<p>Added an example showing what Conduit look likes using a dividing line to differentiate dataproviders.</p>
</li>
<li>
<p>The theme swatch utility has been moved to github</p>
</li>
</ul>GUADEC Report: Late As Usual2008-07-17T09:34:40+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-07-17:2008/07/17/guadec-report-late-as-usual/<p><a href="http://guadec.expectnation.com/guadec08">GUADEC</a> was effing awesome. I have successfully repayed my sleep debt, and can reflect on all that I observed and learned for the week. In bullet point form;</p>
<p><img alt="Insurance to Wake Me Up" src="http://www.johnstowers.co.nz/blog/static/images/imported/cimg0232resized.JPG" />
_Insurance for the last night (WAKE ME UP, English and Turkish) _</p>
<ul>
<li>
<p><a href="http://guadec.expectnation.com/guadec08/public/schedule/detail/25">My Conduit talk</a> went really well. I thought I was able to reach a good balance between 'this is cool for users', 'this is cool for developers', and 'this is a flagrantly useless technical demo because I can'. Following the talk, and over the rest of the week I had a number of chats with people regarding</p>
<ul>
<li>
<p>Improvement hints for the UI. Thanks a lot Bastien and co.</p>
</li>
<li>
<p>What is considered a useful, and achievable, level of mobile device support.</p>
</li>
<li>
<p>Some technical clarifications on matters how to integrate into the desktop (webkit, gio, ui-via-dbus, etc)</p>
</li>
<li>
<p><a href="http://files.conduit-project.org/Conduit%20-%20GUADEC%202008.pdf">Slides available here</a></p>
</li>
</ul>
</li>
<li>
<p>I felt the reception to <a href="http://www.conduit-project.org">Conduit</a> was positive, and I look forward to the module inclusion discussions for 2.24.</p>
</li>
<li>
<p>I was not productive on my summer of code project for that week, but I have come back with an overflowing amount of motivation to not only complete my work on binding syncml, but to also continue to add improvements that will help Conduit operate <em><strong>on</strong></em> Maemo and iPhone, and operate really well <em><strong>with</strong></em> Windows mobile devices, and Nokia devices (via gnome-phone-manager).</p>
<ul>
<li>Dear Team (aka lazyweb): I have quite a large conduit branch I am working on. Whats the plan with bzr-playground.gnome.org. Can I easily push my existing branch there?</li>
</ul>
</li>
<li>
<p>It was great to meet some of the people I have been corresponding with over email, and have never actually seen. You know who you are!</p>
</li>
<li>
<p><a href="http://www.electronicafest.com/">Electronica Festival Istanbul</a> was a great adventure. Thanks to all that made it happen</p>
</li>
<li>
<p>Thanks to the GUADEC volunteers.</p>
</li>
<li>
<p>The 'tab meme' on planet.gnome.org was pretty funny. I was most <a href="http://davyd.livejournal.com/255065.html">fooled by Davyds</a>. (am I the only one who kinda likes this metaphor?). I was also fooled in Federico's key note, when he proposed the NET_WM_I_CAN_HAS_TABS window hint for easy window -> tab transitioning (just kidding).</p>
</li>
<li>
<p>Captain common sense awards are hereby given to:</p>
<ul>
<li>
<p>Team Gtk+: For the reasoned future of Gtk+ discussion/keynote. While I was disappointed to learn aboutthe lack of canvas plans, it is not really surprising. I use Goocanvas, but I am hardly a power user, and without a chosen Canvas, I have zero motivation to switch. I would describe the Gtk3.0+ plan as necessary+pragmatic. That is a damn site better than what it has appeard to be over the last few years (stagnant+idealistic).</p>
</li>
<li>
<p>Realease Time: Nice reveal on the Gnome 2.30 -> Gnome 3.0 animation</p>
</li>
</ul>
</li>
<li>
<p>The <strong>BZR vs. git</strong> thing seemed to reach a whole new level of weirdness at GUADEC. First there was team BZR, whom I shall liken to a Turkish taxi driver. They seem to promise everything, they have the hard sell on, and you can never really be sure how much the cost will be. The git team seems to be like the seasoned traveller. They are showing the finger to the taxi drivers, and generally alternate between cursing at the incompetence of the taxi service, and ignoring them with glee. Instead of discussing the price with the taxi drivers, they instead have a prepared presentation on how flying cars are actually a million times better than taxis because they look so hot.</p>
<ul>
<li>
<p>I actually dont really care what I use. Im not a power user. If I could somehow pay some sysadmin money to setup a unified GNOME themed viewvc that tracked both git and bzr I would. Thats all I want - to be able to track and see what folks are up to. I wrote more about this subject<a href="http://www.johnstowers.co.nz/blog/index.php/2008/05/31/soc-report-week-1/"> here.</a></p>
</li>
<li>
<p>I also support the notion of 'Pick the system that makes it easiest for the distributor to work with'. That likely means git. As a thought experiment, what might be the result of choosing git, or more correctly, rejecting bzr, in the wake of the Ubuntu is moving to KDE fear meme.</p>
</li>
</ul>
</li>
<li>
<p>I Didnt realize the enormous (positive) influence that Nokia has on GNOME. Positive influence is a polite way of saying 'how much money they spend on Gtk+ related subcontractors, and employees'. Apparently something cool is coming for Maemo $NEXT_VERSION</p>
</li>
<li>
<p>If I have forgotten anyone, apologies. See you all next GUADEC!</p>
</li>
<li>
<p>Does anyone look like their Hackergotchi? (and can someone make me one - if you have a photo of me, and its moderately acceptable...)</p>
</li>
<li>
<p>Funniest moments would have to be</p>
<ul>
<li>
<p><a href="http://blogs.gnome.org/johncarr">John Carr</a> with a rats tail and silver necklace!</p>
</li>
<li>
<p>Me and <a href="http://mark.doffman.com/">Marks</a> completely ineffective adventure from one side of Istanbul to the other, with no phone, no watch, no computers, no idea where anyone is, and death by hangover.</p>
</li>
<li>
<p>The humiliation/irony of singing really bad English Kareoke in a country whose official language is Turkish.</p>
</li>
</ul>
</li>
<li>
<p>I have some interesting Conduit work going to land soon</p>
<ul>
<li>
<p>Officially supported Windows Mobile sync support</p>
</li>
<li>
<p>Nokia phone support via gnome-phone-manager</p>
</li>
<li>
<p>Massive refactoring to improve platform abstractions</p>
</li>
<li>
<p>Which includes a move to GIO</p>
</li>
<li>
<p>Which adds a pygtkwebkit based browser</p>
</li>
<li>
<p>Which removes the hard dependency on gnome-keyring</p>
</li>
<li>
<p>Which adds and native python file/vfs abstraction</p>
</li>
<li>
<p>These improve performance and make it a billion times easier to run Conduit on Windows and Mac.</p>
</li>
</ul>
</li>
</ul>SOC Report: Week 12008-05-31T16:13:55+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-05-31:2008/05/31/soc-report-week-1/<p>Well the first week of summer of code has finished. This week I spend my time evaluating and testing the various options available to (semi)automatically wrap C code (<a href="http://libsyncml.opensync.org/">libsyncml</a>) so that it is accessible from Python. My priorities when evaluating the options go something like,</p>
<ol>
<li>
<p>Capability - the tool should be able to (semi) automatically wrap a large majority of the libsyncml api. Any customizations required in order to make the wrapping more complete should be readable and maintainable by people other than myself.</p>
</li>
<li>
<p>Documentation availablitly. Follows from #1, can I actually learn and use the tool within the SOC duration.</p>
</li>
<li>
<p>The wrapping tool is actively developed.</p>
</li>
<li>
<p>Does not introduce additional runtime dependencies other than the library being wrapped.</p>
</li>
<li>
<p>Minimal compile time dependencies when creating the bindings.</p>
</li>
<li>
<p>Community service value (i.e does the selection and use of the tool bring a positive benefit to the FOSS ecosystem greater than the actual library being wrapped).</p>
</li>
</ol>
<p>The following is a list of available options I looked at (see <a href="http://wiki.cython.org/WrappingCorCpp">cython</a> for more explanation)</p>
<ul>
<li>
<p><a href="http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/">Pyrex
</a>Produce very nice and clean C file, which you just compile to .so and that's it. Allows to wrap almost any C and C++ code. IDL is python-ish.</p>
</li>
<li>
<p><a href="http://cython.org/">Cython
</a>The same as Pyrex, but some new nice features added</p>
</li>
<li>
<p><a href="http://www.swig.org/">SWIG
</a>The defacto standard I guess. SWIG is one of the oldest and most mature methods of wrapping C or C++ code into Python (SWIG works for other target languages as well). SWIG produces a C file from an IDL, which gets compiled to a .so, but then it also produces a Python wrapper on top of this. Because Python wrappers are written for you, if their design is not exactly what you want, you end up doing more work to create your final Python API.</p>
</li>
<li>
<p><a href="http://www.riverbankcomputing.co.uk/sip/">SIP</a>
Similar to SWIG, but only aimed at wrapping C and C++ to Python. Unlike SWIG there is no Python wrapper. Used by PyQT and PyKDE.</p>
</li>
<li>
<p><a href="http://www.boost.org/libs/python/doc/">Boost.Python</a>
Writes C++. Not evaluated due to the additional dependencies required.</p>
</li>
<li>
<p>Ctypes
Ctypes is included standard in Python 2.5. The IDL is typically a python class hiding the ctypes calls, making the API more pythonic. It allows one to call library functions defined in shared object libraries directory from interpreted Python code.</p>
</li>
<li>
<p><a href="http://www.language-binding.net/pyplusplus/pyplusplus.html">Py++</a>
It generates Boost.Python wrappers. Not evaluated.</p>
</li>
<li>
<p><a href="http://www.scipy.org/F2py">f2py</a>
It's mostly for wrapping fortran files, but it can also wrap C files, even though it's not a very well-known feature. Not evaluated</p>
</li>
<li>
<p><a href="http://pyd.dsource.org/">PyD</a>
This works like boost.python, but for the D language. Not evaluated.</p>
</li>
<li>
<p><a href="http://panda3d.net/wiki/index.php/Interrogate">Interrogate</a>
This works similar to SWIG. It created dynamic link libraries that can be used both from python and c++ via the Python C API. No other files are needed. Its not very well documented but is used in several commercial mmorpg's and is native to the Panda3d engine. Not evaluated.</p>
</li>
<li>
<p><a href="http://robin.python-hosting.com/">Robin
</a>Insufficient documentation to evaluate.Similar approact to swig, sans the intermediate IDL.</p>
</li>
<li>
<p><a href="https://code.launchpad.net/pybindgen">PyBindgen</a>
The IDL is itself python, and it generates clean readable dependency free C code. Designed for wrapping C++, but has some support for wrapping C libs.</p>
</li>
<li>
<p><a href="http://svn.gnome.org/viewvc/pygobject/trunk/codegen/">pygobject (codegen.py and h2defs.py)</a>
The Gobject way, and the way I am most familiar. Unfortunately, in order to wrap the libsyncml library I would first need to wrape it in GObject.</p>
</li>
</ul>
<p><strong>Conclusions</strong></p>
<p>The libsyncml library uses the Gobject mainloop, and custom error types. In order to integrate this with pygtk applications It would need to link to Pygobject/C, and propogate the error types to exceptions.</p>
<p>Somewhat unsurprisingly, the weak point in almost all of these approaches is there documentation. While I like the look of PyBindgen, it is a nightmare to build, and docs are sparse. The SWIG IDL is hairy, and one must also maintain pythonic wrappers to make a nice library. Pyrex and friends do not seem suited to the integration of libsyncml and pygobject without additional C glue</p>
<p>At this stage I am leaning towards SWIG, for community service value (others can come along after and make C# wrappers for instance), its availability of documentation, and even if the IDL is quirky, others are familiar with it.</p>
<p><strong>Distributed Version Control Systems and visibility of development</strong></p>
<p>My opinion on the 'best' DVCS is not relevant. What I am concerned about is that if GNOME does not pick one, and/or provide some sort of hosting or method to track other peoples development branches then the visible activity level, and subsequently health of the whole project will suffer.</p>
<p>The premise here is that centralized version control systems make it easy to follow what developers are working one, and the activity level of development, via the <a href="http://mail.gnome.org/mailman/listinfo/svn-commits-list">svn-commits mailing list</a> for example.</p>
<p>I can only offer anecdotal evidence here, but I think that the visibility a projects development is just as important as the actual rate of development being done.</p>
<ul>
<li>
<p>If developers cannot see what other people are hacking on, then there is the potential for duplication of work, or conflicting implementations.</p>
</li>
<li>
<p>If users do not see people actually doing work, then there is a tendency to assume the project is 'abandoned' or dead. The only thing worst than a 'dead' project is being proclaimed as such when one is not.</p>
</li>
</ul>
<p>I consider the plethora of ways one can follow what developers are doing part of the problem, not part of the solution.</p>
<p>Who has time to follow planet, IRC, github, repo.or.cz, freedesktop git, launchpad.net bzr, mailing lists, twitter, $COMPANY gitweb, $PERSONAL gitweb, $DISTRO viewvc and gnome.org/$USER_HOME_DIR to see what people are working on. </p>
<p>This post is not meant to be Reductio ad absurdum, its just a slight generalization of why I read planet.gnome.org/svn-commits, etc, etc.</p>
<ul>
<li>
<p>Part of the reason is to see what other hackers are up to.</p>
</li>
<li>
<p>Part is academic, to learn techniques and design from some of the great hackers on here.</p>
</li>
<li>
<p>Part is flagrant procrastination.</p>
</li>
<li>
<p>The small remaining part is the keep the voice in my head thats says "you should be using KDE, it appears to be more actively developed" at bay.</p>
</li>
</ul>
<p>Conclusions, if any;</p>
<ul>
<li>
<p>planet.ubuntu.com seems to have excellent visibility of active development, even if it doesn't have as many developers as other distributions.</p>
</li>
<li>
<p>freedesktop (via planet.freedesktop.org and http://gitweb.freedesktop.org/) seems to have excellent visibility of development (many people put git branches in their home directories, which are subsequently picked up by gitweb).</p>
</li>
<li>
<p>I am not advocating activity over productivity (obviously we are all free to use the tools which allow us to be the most productive, not just appear the most active). I just think that public FOSS development is an interesting space, in many ways the developers of the products are the marketers of it.</p>
</li>
<li>
<p>GNOME used to have the balance of visibility about right, but I think we are losing that with all this dilution.</p>
</li>
</ul>
<p>Change scares me. That is all.</p>Frantic2008-05-21T20:22:38+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-05-21:2008/05/21/frantic/<p>Frantic would be how I described my last two weeks. I have had very little free time to work on Conduit. Everything seems to have come at once!</p>
<ul>
<li>
<p>I got accepted for summer of code - <a href="http://code.google.com/soc/2008/gnome/appinfo.html?csaid=6D8BB0A85489E843">adding SyncML support to Conduit.</a></p>
</li>
<li>
<p>Alexandre Inacio Rosenfeld also got accepted to SOC to work on <a href="http://code.google.com/soc/2008/gnome/appinfo.html?csaid=90F643A2BBAEC3D7">Ipod and media support in Conduit.</a></p>
</li>
<li>
<p>I got accepted to <a href="http://guadec.expectnation.com/guadec08/public/schedule/detail/25">speak at GUADEC</a>, which I will be attending, providing all finances work out.</p>
</li>
<li>
<p>It looks likely that, as part of my PhD, I will be spending a few months at <a href="http://www.enac.fr/">ENAC</a> (home of the <a href="http://paparazzi.enac.fr/">Paparazzi</a> UAV).</p>
</li>
</ul>
<p><strong>Random</strong></p>
<ul>
<li>
<p>I have been playing with <a href="http://www.igelle.org/barpanel/">barpanel</a>, a very functional GNOME panel replacement.</p>
</li>
<li>
<p><a href="http://tayasui.com/Grape.html">Grape</a> is certainly an interesting UI/desktop mock up. If I had infinite spare time I migh have a hack on it, as an excuse to play with <a href="http://www.clutter-project.org">Clutter.</a></p>
</li>
<li>
<p>Props to Jan Bodnar for his excellent <a href="http://zetcode.com/tutorials/gtktutorial/">Gtk+</a> and <a href="http://zetcode.com/tutorials/cairographicstutorial">Cairo</a> tutorials.</p>
</li>
<li>
<p>My (bad) experiences with Ubuntu 8.04 can be best described by the following picture..
<img alt="Firefox crashing" src="http://www.johnstowers.co.nz/blog/static/images/imported/ffcrash.png" /></p>
</li>
</ul>
<p><strong>Openstreetmap GPS Mapping Widget</strong></p>
<p>Somewhat tangentially related to my PhD, I have been hacking on a <a href="http://nzjrs.github.com/osm-gps-map">simple Openstreetmap GPS mapping/display widget</a>. Basically because after investigating all the existing mapping programs on linux, none of them supported openstreetmap/openaerialmap and were able to be easily embedded.</p>
<p><img alt="OSM GPS map widget" src="http://www.johnstowers.co.nz/blog/static/images/imported/osm-gps-map.png" /></p>
<p>It's basically a port of<a href="http://www.tangogps.org/gps/cat/News"> tangoGPS (by Marcus Bauer)</a> to libsoup and considerable clean-up. The whole thing is now hidden behind a derived GtkDrawingArea with a nice simple 4 function api (other parameters such as zoom, lat, lon, are accessible as gobject properties)</p>
<ol>
<li>
<p>set_map_center(double lat, double lon, int zoom)</p>
</li>
<li>
<p>add_gps_point(double lat, double lon)</p>
</li>
<li>
<p>add_roi(double lat, double lon, GdkPixbuf *pixbuf)</p>
</li>
<li>
<p>get_bouding_box()</p>
</li>
</ol>
<p>Things like double click, map dragging, scroll to zoom, etc are all handled automatically as you would expect. It caches downloaded tiles and it's pretty much complete at this point. I hope to be able to post code soon.</p>More Conduit GSOC Ideas2008-04-01T11:33:21+02:00John Stowerstag:www.johnstowers.co.nz/blog,2008-04-01:2008/04/01/more-conduit-gsoc-ideas/<p>I see that Google has extended the SOC application deadline. Here are some Conduit related <a href="http://live.gnome.org/SummerOfCode2008/Ideas/Conduit">SOC ideas</a> for GNOME.</p>
<ul>
<li>
<p>Port Tomboy sync to use Conduit (and get free support for $WEBSERVICES)
Use Conduits DBus interface and our C# bindings to said interface to be able to configure and initiate synchronization from Tomboy. This means that peer-to-peer Tomboy sync will get easier (no more ssh fuse), and that support for additional websites/mobile devices will become available. </p>
</li>
<li>
<p>Port F-Spot photo export to use Conduit
With the exception of mentaloo gallery, we support most/all of what F-spot can export to. I can count a number of times where F-spot has hung/crashed during photo export. Using Conduit to do the sync will help prevent this, and ensure that if something untoward does happen, when you restart the sync duplicate photos will not be uploaded. </p>
</li>
<li>
<p>Mobile phone support </p>
<ul>
<li>
<p>Integration with gnome-phone-manager for phone discovery and/or pim data Get/Set </p>
</li>
<li>
<p>Use of python-gammu for fetching PIM data, and photos from those devices that do not support obex-ftp </p>
</li>
</ul>
</li>
<li>
<p>Gstreamer based media transcoding
We currently call FFMPEG or mencoder via command line to convert/scale video and audio files. I would like to use gstreamer. A possible solution to this would be to create a gstremer transcoding utility (GUI and command line), or the use of the python gstreamer bindings from within Conduit. While I prefer the latter, the former would be a useful addition to the GNOME desktop. </p>
</li>
<li>
<p>Port cheese to use Conduit for photo and video site upload
This would be based upon out glib dbus bindings. </p>
</li>
<li>
<p>More GNOME plugins using out DBus interface </p>
<ul>
<li>
<p>(finish) Eog plugin for photo site upload </p>
</li>
<li>
<p>Add video upload to Youtube and Vimeo from Totem </p>
</li>
<li>
<p>Better nautilus integration. Removable volume support has improved in Conduit. It would be good to expose this from nautilus, although I am not sure the role this would take, for example </p>
</li>
</ul>
</li>
<li>
<p>Support windows mobile devices. We have preliminary support for SyncCE, but this was never completed. There are capable python bindings to SyncCE, so it would take a hacker with a WM5/WM6 device to finish this. </p>
</li>
<li>
<p><a href="http://bugzilla.gnome.org/show_bug.cgi?id=510113">Support palm pilots</a>. Once again, not something I can work on as I do not have a device. There are some python bindings for getting data from Palm devices, and there is also the possibility of wrapping the GNOME pilot code to enable it to be used from Python.</p>
</li>
</ul>New Conduit Features2008-03-29T14:35:15+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-03-29:2008/03/29/new-conduit-features/<p>Last week (or so..) I released <a href="http://www.conduit-project.org/wiki/0.3.9">Conduit 0.3.9</a>, the notable features of this release were the addition of documentation, and dramatically improved support for removable devices, like USB keys and portable hard drives.</p>
<p><img alt="Conduit Documentation" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-help.png" /></p>
<p>Conduit has always supported removable disks, but the UI for working with them has been dramatically improved. In the screenshot below you will see Conduit automatically suggesting pre-configured dataproviders to sync with folders on the removable device. The use case for this is</p>
<ol>
<li>
<p>Joe synchronizes stuff from one PC to his USB key.</p>
</li>
<li>
<p>When Joe plugs that USB key into another computer also running Conduit he is presented with a pre-configured method to synchronize information from the usb key to a location on his local machine.
<img alt="Preconfigured USB key and mobile phone discovery" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-usb-and-phone.png" /></p>
</li>
</ol>
<p>The functionality is not all that different to before (it shares all the same code as the traditional folder <--> folder sync, meaning you get conflict detection etc) but the way it is now presented is more sensible and discoverable.</p>
<p>In the screenshot you will also see the first commit of <a href="http://bugzilla.gnome.org/show_bug.cgi?id=520668">mobile phone support</a> in Conduit. Phone discovery is done using bluetooth. Picture sync is implemented atop gnomevfs-obex-ftp. Data is currently fetched from the phone using <a href="http://cihar.com/gammu/python/">gammu</a>, but <a href="http://bugzilla.gnome.org/show_bug.cgi?id=456259">discussion are underway </a>on how this might tie into gnome-phone-manager (which uses gnokii).</p>
<p>This work is (mildly) separate, but necessary for a sensible SyncML (using <a href="http://libsyncml.opensync.org">libsyncml</a>) implementation. In order to provide smart suggestions on what sync options to offer for a particular phone, phones in range (via bluetooth or USB connection) should be enumerated and their capabilities queried. Currently I am using bluez for the former, and gammu for the latter, however, at least in the bluetooth case, the latter could probbably be accomplished using pure AT commands over a serial socket to get the manufacturer and model number and them comparing this with a database of capabilities.</p>
<p>If its not clear, this is still in the experimental stage, but the current approach seems to demonstrate that Conduits architecture, and the approach to implementation are approximately correct.</p>
<p>There have also been a few other fixes go into SVN, including</p>
<ul>
<li>
<p>The ability to create tags in F-spot from within the Conduit UI</p>
</li>
<li>
<p>Improved configuration for picasa web galleries</p>
</li>
</ul>
<p>Finally I will finish with a dear lazyweb.....</p>
<p><em>Dear Lazyweb; The current implementation of network sync doesnt really scale to pushing large files around. Its using the python pickle module, which has a tendency to read everything into memory, base64 encode it, run out of memory, and then explode. I have considered may ways to solve this</em></p>
<ul>
<li>
<p><em>Use sftp - but that requires the user setup a ssh keypair with the other computer. Seahorse, can you give me a dbus interface to make this happen, easily?</em></p>
</li>
<li>
<p><em>Re-implement (because the license is CPL), some of the <a href="http://cvs.rpath.com/conary-docs/conary.lib.util-pysrc.html">ingenious performance optimizations in conary</a> WRT decreasing XMLRPC memory usage.</em></p>
</li>
<li>
<p><em>Wait for the GNOME desktop to offer some sort of GiveFileToThisUser() dbus api call. Could perhaps be implemented atop Giver/Telepathy (if... it gets accepted to the desktop....)</em></p>
</li>
<li>
<p>_Custom python module written in C to do this.... _</p>
</li>
<li>
<p><em>Continue doing what I am doing, and wait for people to complain louder....</em></p>
</li>
</ul>Why Engineering and FOSS is Satisfying2008-03-28T20:56:34+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-03-28:2008/03/28/why-engineering-and-foss-is-satisfying/<p>I recently came across the post; <a href="http://drewyates.net/why-student-programmers-rant-about-business-students-with-ideas">why student programmers rant about business students with ideas</a>. Putting my ideological belief in FOSS aside, I think it eloquently describes why I chose to return and pursue a PhD after a year spent studying engineering management (like an MBA - for those North American folks).</p>
<p>I also think it is highly applicable to FOSS. Many successful FOSS projects are born from engineers, the <em>'decision to execute'</em> has already been made. As the article mentions, <em>'leadership and the ability to make decisions _is _valuable, but only in groups with realizable ability to execute'</em>. As a FOSS project evolves, the (normal?) combination of a BFDL and the constant freedom to fork keep the project relatively free of the ownership style disputes described in the post.</p>
<p>In reading the post and its comments, its also refreshing to see a discussion on the internet so free of rudeness.</p>
<p>[1] The terms engineer and programmer can be used interchangeably, in this context. They are both highly skilled, creative professions, commonly only recognized as such by their respective peers.
[2] Im obviously not so naive as to rule out the influence of money on all of this. I just consider it a baseline, and a gross generalization, that monetary incentives are less important to creative professions than to those who pursue an MBA.</p>Summer of Code2008-03-13T11:22:29+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-03-13:2008/03/13/summer-of-code/<p>Work on Conduit is progressing nicely, squashing bugs that have appeared when interacting with the new versions of GNOME applications. I have kind of been tracking the GNOME release schedule (congratulations on the latest release BTW), so I expect to put out another Conduit release this week.</p>
<p>I intend to keep making stable bug fix releases until there are Python GIO bindings. At that time it would probably make sense to branch so that I can begin targeting 2.24, and have the opportunity to land some more invasive changes.</p>
<p>The current bugs that are being hacked on (i.e. have patches), depending on how the implementation turns out, may appear in $NEXT_STABLE_RELEASE, or in $FUTURE_RELEASE.</p>
<ul>
<li>
<p><a href="http://bugzilla.gnome.org/show_bug.cgi?id=521196">#521196 Sync FSpot photos with changed captions/tags</a></p>
</li>
<li>
<p><a href="http://bugzilla.gnome.org/show_bug.cgi?id=511691">#511691 Improve removable volume/folder support</a></p>
</li>
<li>
<p><a href="http://bugzilla.gnome.org/show_bug.cgi?id=520668">#520668 Mobile phone support</a></p>
</li>
<li>
<p><a href="http://bugzilla.gnome.org/show_bug.cgi?id=520699">#520699 Support Google contacts using gdata</a></p>
</li>
</ul>
<p><strong>Summer of Code Projects</strong>
I create <a href="http://live.gnome.org/SummerOfCode2008/Ideas/Conduit">this page on the GNOME wiki</a> which includes some summer of code ideas for Conduit (and by extension, synchronization in the GNOME desktop).</p>
<p>Disclaimers:</p>
<ol>
<li>
<p>I am eligible, and will be applying to participate in summer of code this year.</p>
</li>
<li>
<p>I wont be offended if people apply to do the same tasks that I am going to be proposing. The more the merrier!</p>
</li>
<li>
<p>Im not sure how the whole mentoring regulations work. Can I mentor and be a student?, are there mentors in the GNOME community who would like to champion Conduit integration in their application?</p>
</li>
</ol>Coming Soon to an Internet Tablet Near You2008-02-22T18:14:19+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-02-22:2008/02/22/coming-soon-to-an-internet-tablet-near-you/<p><img alt="Conduit on Maemo" src="http://www.johnstowers.co.nz/blog/static/images/imported/dsc01791.JPG" /></p>
<ul>
<li>
<p>Multiple site photo upload</p>
</li>
<li>
<p>PIM sync to your desktop (including over the air sync using our network sync support)</p>
</li>
<li>
<p>File/Folder sync to your desktop and to online services such as box.net</p>
</li>
<li>
<p>The same codebase as Conduit for the desktop</p>
</li>
<li>
<p>The awesome work of <a href="http://picasaweb.google.com/thomas.vanmachelen/Conduit">Thomas Van Machelen</a></p>
</li>
<li>
<p>Did I say shiny?</p>
</li>
</ul>They come in pairs2008-02-18T23:28:42+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-02-18:2008/02/18/they-come-in-pairs/<p>Following the tradition of the last Conduit release, I realized that I shipped Conduit 0.3.7 with a stupid build issue that broke all the Gmail/Google/Picasa support for people. Following in that tradition here is Conduit 0.3.8 which fixes that, and also features updated icons contributed by mejogid.</p>
<p><img alt="New Conduit Icons" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-new-icons.png" /></p>
<p>For more information:</p>
<ul>
<li>
<p><a href="http://www.conduit-project.org/wiki/0.3.8">Release Notes</a></p>
</li>
<li>
<p><a href="http://download.gnome.org/sources/conduit/0.3/">Download</a></p>
</li>
</ul>
<p>Conduit developer John Carr has also setup a <a href="https://launchpad.net/~conduit/+archive">Ubuntu PPA for Conduit</a>. Check it out if you are interested in running the Latest and Greatest for Ubuntu Gutsy and Hardy.</p>Conduit 0.3.7: Many Small Updates2008-02-14T20:04:39+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-02-14:2008/02/14/conduit-037-many-small-updates/<p>I just released <a href="http://www.conduit-project.org/wiki/0.3.7">Conduit 0.3.7</a>. This was another regular release which contains the usual array of bug fixes and a few small new features and tidy ups. This was also the first release made automatically using <a href="http://developer.imendio.com/projects/misc/maintainer">maintainer</a>. Aside from a few quirks, the tool worked amazingly well.</p>
<p>The next release will feature <a href="http://live.gnome.org/Conduit/Documentation">rewritten</a> user documentation, more conflict UI improvements, and fix whatever bugs crop up in testing its compatibility with new GNOME. Highlights of this release are;</p>
<ul>
<li>
<p>Update to use the newer Tomboy Set/GetNoteComplete API. This finally allows lossless sync of Tomboy notes including metadata such as tags and notebooks.</p>
</li>
<li>
<p>Update to the latest versions of our shipped static libraries</p>
<ul>
<li>
<p>flickrapi (fixes many Flickr bugs)</p>
</li>
<li>
<p>pyfacebook</p>
</li>
<li>
<p>python-gdata (and use this for Picasa support)</p>
</li>
<li>
<p>libgmail (make emails work again)</p>
</li>
</ul>
</li>
<li>
<p>The Eye of gnome plugin for uploading photos to multiple sites now shows status while synchronizing</p>
</li>
<li>
<p>Add one-way support for syncing Gmail contacts (with evolution, ipod, etc)</p>
</li>
<li>
<p>Add a 'developer' sub-menu to the help menu if the user is running a development version. This sub-menu contains items which link to the <a href="http://www.conduit-project.org/wiki/Development">newly improved developer documentation</a>. The linked pages are displayed in conduit's built in web browser.</p>
</li>
<li>
<p>Make the Conflict preferences per Conduit add descriptive icons to the preferences menu</p>
</li>
</ul>
<p><img alt="Conduit 0.3.7: Many Small Updates" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit037xparent.png" /></p>
<p><strong>Update:</strong> I completely forgot. Thomas Van Machelen recently added C# bindings for our DBus API to SVN. If any of you enterprising Novell Hack Week hackers wants to play with them, I have some ideas (use Conduit to export photos from FSpot, use Conduit as the sync engine for Tomboy, for both synchronization of notes between two computers and/or export to ipod, box.net, etc). Please email me if you are interested. I know Conduit is ready, there are many examples of the Conduit DBus API in SVN.</p>Linux.conf.au Talk2008-01-26T14:36:42+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-01-26:2008/01/26/linuxconfau-talk/<p>Just a quick note to say that I'm giving a talk on Conduit at 9:20am Tuesday 29th of January at <a href="http://live.gnome.org/Melbourne2008">GNOME.conf.au</a>. Come along if you are interested. I will be going through some demos showing the many ways you can use Conduit including;</p>
<ul>
<li>
<p>As a multiple site photo uploader</p>
</li>
<li>
<p>To keep your Tomboy notes in sync</p>
</li>
<li>
<p>As a file/folder sync tool</p>
</li>
<li>
<p>Keep your GConf preferences in sync over multiple computers</p>
</li>
<li>
<p>Keeping two computers in sync over the local network</p>
</li>
<li>
<p>Sycing with removable devices like Nokia n800 and iPods</p>
</li>
<li>
<p>As a PIM sync tool supporting a number of online services</p>
</li>
</ul>
<p>I will also be demonstrating how to write your own dataprovider, and hopefully I will have time to discuss more difficult issues such as</p>
<ul>
<li>
<p>Furthur integration with GNOME (Roadmap to 2.24)</p>
</li>
<li>
<p>SynCE/Opensync/SyncML</p>
</li>
<li>
<p>Progress on the Maemo/Hildon port</p>
</li>
</ul>Shape of things to come2008-01-19T19:23:53+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-01-19:2008/01/19/shape-of-things-to-come/<p>I just <a href="http://svn.gnome.org/viewvc/conduit?view=revision&revision=1249">committed</a> the first version of a working GObject binding to the <a href="http://www.conduit-project.org">Conduit</a> DBus API. You can check out an example test program <a href="http://svn.gnome.org/viewvc/conduit/trunk/conduit-glib/test/test-application.c?view=markup">here</a>.</p>
<h3>Contrived Example-like Code</h3>
<div class="codehilite"><pre><span></span> /* The running conduit app/daemon */
ConduitGApplication *application = conduit_g_application_new ();
/* Get two dataproviders, a datasource and datasink */
ConduitGDataprovider *source = conduit_g_application_get_dataprovider(application, "TestSource");
ConduitGDataprovider *sink = conduit_g_application_get_dataprovider(application, "TestSink");
/* Put them in a conduit so we can sync them */
ConduitGConduit *conduit = conduit_g_application_build_conduit(application,source,sink);
g_signal_connect (conduit, "sync-progress", (GCallback) _sync_progress_cb, NULL);
g_signal_connect (conduit, "sync-completed", (GCallback) _sync_completed_cb, NULL);
g_signal_connect (conduit, "sync-conflict", (GCallback) _sync_conflict_cb, NULL);
/* Add it to the GUI so the user can see it (optional) */
ConduitGSyncset *gui_syncset = conduit_g_syncset_new(application, "/syncset/gui");
conduit_g_syncset_add_conduit(gui_syncset, conduit);
/* Perform the sync (boom!) */
conduit_g_conduit_sync(conduit);
g_main_loop_run (mainloop);
</pre></div>
<h3>Conclusion</h3>
<ul>
<li>
<p>A GObject binding to the Conduit DBus API means you can synchronize your application data, from within your application, in very few lines of code;</p>
<ul>
<li>
<p>Your application is not going to crash</p>
</li>
<li>
<p>Your application is not going to hang or block the UI</p>
</li>
<li>
<p>The difficult sync logic for interfacing with various other places can be written in Python (faster, more productive, most web-services get python bindings first), live in the Conduit source tree, and immediately be available to all other applications which use Conduit to sync stuff.</p>
</li>
</ul>
</li>
<li>
<p>In SVN NOW!</p>
</li>
<li>
<p>Hat-tip to Mikkel Kamstrup Erlandsen whose <a href="http://xesam.org/people/kamstrup/xesam-glib/">Xesam Glib bindings</a> provided a useful starting point, and saved me much autotools pain.</p>
</li>
<li>
<p>Yes, the names are a bit long.Who has a better idea for the prefix, gcond_ perhaps?</p>
</li>
<li>
<p>Who wants to help me integrate this into;</p>
<ul>
<li>
<p>Cheese</p>
</li>
<li>
<p>Evolution</p>
</li>
<li>
<p>GNOME Phone Manager</p>
</li>
<li>
<p>$YOUR_APP_HERE</p>
</li>
</ul>
</li>
</ul>Conduit 0.3.6: I can has sleep please?2008-01-17T15:56:07+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-01-17:2008/01/17/conduit-036-i-can-has-sleep-please/<p>Quote courtesy of <a href="http://unrouted.co.uk/">John Carr</a>, who I had cook me up a patch at 1:30AM UK time. Ahhh the wonder of a 12h time difference.</p>
<p>I just released Conduit 0.3.6, a quick follow up to the <a href="http://www.johnstowers.co.nz/blog/index.php/2008/01/16/back-to-reality/">last release</a>. It fixes a number of bugs which were preventing some common sync scenarios.</p>
<ul>
<li>
<p><a href="http://download.gnome.org/sources/conduit/0.3/">Download</a></p>
</li>
<li>
<p><a href="http://www.conduit-project.org/wiki/0.3.6">Release Notes</a></p>
</li>
<li>
<p><a href="http://bugzilla.gnome.org/enter_bug.cgi?product=conduit">Report Bugs</a></p>
</li>
<li>
<p><a href="http://mail.gnome.org/mailman/listinfo/conduit-list">Mailing List</a></p>
</li>
<li>
<p><a href="http://tests.conduit-project.org/">Test Results</a></p>
</li>
<li>
<p>Bugs Fixed</p>
<ul>
<li>
<p>Fix two-way Tomboy <--> folder sync</p>
</li>
<li>
<p>Fix two-way GConf <--> folder sync</p>
</li>
<li>
<p>Dont recreate the backpack API everytime we refresh</p>
</li>
<li>
<p>Fix Shutterfly to support image uploads and delete again (Jeremy Slater)</p>
</li>
<li>
<p>Fix attribution of Banshee dataprovider. It was contributed by Don Smith</p>
</li>
<li>
<p>Many fixes to image dataproviders</p>
</li>
</ul>
</li>
</ul>
<p><img alt="Always Up To Date Sync" src="http://www.johnstowers.co.nz/blog/static/images/imported/alwaysuptodate.png" /></p>
<p><em>OK I lied, there is one new feature..</em></p>Back To Reality2008-01-16T03:48:45+01:00John Stowerstag:www.johnstowers.co.nz/blog,2008-01-16:2008/01/16/back-to-reality/<h4>Conduit 0.3.5: Enough Excuses</h4>
<p>I am a terrible project maintainer. It has been a full 3 months since the last <a href="http://www.conduit-project.org/">Conduit</a> release, and I have run out of excuses. First I got distracted by online desktop shenanigans, then by <a href="http://fridge.ubuntu.com/node/1211">Opensync and Ubuntu things</a>. I got caught up in moving Conduit to <a href="http://svn.gnome.org/viewvc/conduit/">GNOME SVN</a>. Then I got distracted by <a href="http://www.johnstowers.co.nz/blog/index.php/2007/12/16/jhbuild-anything-on-windows-in-12-steps/">JHBuild on windows</a>. Finally I lost my nerve and went on holiday, not touching a computer for 3 weeks. Enough is enough!</p>
<p>The good news however is that there is no doubt in my mind that this is the best Conduit release ever. I'm using Conduit on a daily basis for a variety of tasks, and its looking really solid. Yes there are still some bugs, but thats life. So, no screenshots this time, just a list of shiny new features. Over the next few weeks I will be doing a series of "look what you can do with conduit" type posts, as this is probably the best way to demonstrate to folks what they can accomplish with a generic sync+conversion framework, with a focus on integration with the desktop environment and great support of online services!</p>
<ul>
<li>
<p><a href="http://www.conduit-project.org/wiki/0.3.5">Release Notes</a></p>
</li>
<li>
<p><a href="http://download.gnome.org/sources/conduit/0.3/">Download</a></p>
</li>
<li>
<p><a href="http://bugzilla.gnome.org/enter_bug.cgi?product=conduit">Report Bugs</a></p>
</li>
<li>
<p>Whats New</p>
<ul>
<li>
<p>Move to <a href="http://svn.gnome.org/viewvc/conduit/">GNOME SVN</a> and <a href="http://bugzilla.gnome.org/browse.cgi?product=conduit">Bugzilla</a></p>
</li>
<li>
<p>In memory mapping DB has been replace with sqlite based one. Should reduce memory usage during large sync's.</p>
</li>
<li>
<p>Restructure code layout to allow Conduit to be used as a library by other python programs.</p>
</li>
<li>
<p>Removed Gtk dependancy in core. Conduit command line can now be run without Gtk installed.</p>
</li>
<li>
<p>Added 'always up to date' infrastructure for automatic syncing.</p>
</li>
<li>
<p>Improved n800 support for syncing audio, video and photos to the device.</p>
</li>
<li>
<p>New AudioVideoConverter for transcoding media types (uses FFMPEG and/or mencoder)</p>
</li>
<li>
<p>Include Banshee support in configure.ac - from lool</p>
</li>
<li>
<p>Support data hashes in addition to mtimes as a means for detecting changes</p>
</li>
<li>
<p>Remove twisted dependency for network sync. Uses the built in python xmlrpc module</p>
</li>
<li>
<p>Use python logging module for better logging granularity</p>
</li>
<li>
<p>Make HAL easier to extend in dataprovider factories</p>
</li>
<li>
<p>Files that did not load correctly are shown in the preferences window to help debugging</p>
</li>
<li>
<p>Experimental maemo/hildon UI (Thomas)</p>
</li>
<li>
<p>Added iPod photos support</p>
</li>
<li>
<p>Write support in f-spot using its new dbus interface (Thomas)</p>
</li>
<li>
<p>Support tags in all datatypes (useful for images)</p>
</li>
<li>
<p>Incredible improvements in test coverage. 75% lines of core code tested (including close to 100% of the core sync engine)</p>
</li>
<li>
<p>RSS Feed source now supports video enclosures</p>
</li>
<li>
<p>Many many bug fixes as a result of test focus</p>
</li>
</ul>
</li>
</ul>
<p>As more people start to use conduit I am becoming more conservative and focusing a lot more on testing. One aspect of this is a machine I have set up that runs the test suite every time there is a commit. The other aspect is more regular releases. For the former look <a href="http://tests.conduit-project.org/">here</a>, for the latter please test and give me feedback!</p>
<p>Team conduit is well poised for a 0.4.0 release to coincide with GNOME 2.22.</p>
<h4>Holidays</h4>
<p>It feels strange to be back at university after a solid 3 weeks of doing very little. I had the best summer holiday I have had in a long time. Considerable thanks must go to the weather in New Zealand, its great to finally have the good weather in December. Anyway here are some holiday snaps for those that are curious about what a summer Christmas/New Year in New Zealand looks like.</p>
<p><img alt="My Christmas Present" src="http://www.johnstowers.co.nz/blog/static/images/imported/car-racing-115resized.jpg" />
<em><a href="http://www.fcr.co.nz/">My Christmas Present</a></em></p>
<p><img alt="Relaxing on the Jetty" src="http://www.johnstowers.co.nz/blog/static/images/imported/cimg9761resized.JPG" />
<em>Relaxing on the Jetty</em></p>
<p><img alt="Sunset" src="http://www.johnstowers.co.nz/blog/static/images/imported/p1010860resized.JPG" />
<em>Sunset at the <a href="http://maps.google.co.nz/?ie=UTF8&ll=-41.191444,174.177933&spn=0.080865,0.160675&t=h&z=13&om=1">Bay Of Many Coves</a></em></p>
<p><img alt="Diving" src="http://www.johnstowers.co.nz/blog/static/images/imported/cimg9770resized.JPG" />
<em>Hunter Gatherer</em></p>Jhbuild Anything on Windows - In 12 Steps2007-12-16T10:01:50+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-12-16:2007/12/16/jhbuild-anything-on-windows-in-12-steps/<h2>Part 1: The Complete Gtk+ Stack from Source</h2>
<p>Its been a long week spent watching things compile [0], but I'm happy to report that Jhbuild is now able to build the complete [1] stable Gtk+ stack on windows using the msys/mingw tool chain. That literally means that you can now build you Gtk app, or Gtk itself, for windows, in 12 steps. You still getting all the normal benefits that Jhbuild allows. So, the steps are;</p>
<ol>
<li>
<p>Install <a href="http://sourceforge.net/project/showfiles.php?group_id=2435&package_id=240780&release_id=529741">MinGW combined installer (MinGW-5.1.3) </a>to C:\mingw</p>
</li>
<li>
<p>Untar <a href="http://downloads.sourceforge.net/mingw/msysCORE-1.0.11-2007.01.19-1.tar.bz2?modtime=1169214215&big_mirror=1">msysCORE-1.0.11-2007.01.11</a> to C:\msys</p>
</li>
<li>
<p>Run C:\msys\postinstall\pi.bat, answer yes when it asks whether mingw is already installed and point it to C:\ming</p>
</li>
<li>
<p>Install <a href="http://www.python.org/ftp/python/2.5.1/python-2.5.1.msi">Python 2.5.1</a> to default location (C:\Python25)</p>
</li>
<li>
<p>Install <a href="http://www.activestate.com/">ActivePerl</a> to C:\Perl. You can probably also install a MingW-provided perl package, but the latest version of this was some 5.6 at the time of this writing, while ActivePerl is 5.8.</p>
</li>
<li>
<p>Install <a href="https://launchpad.net/bzr/1.0/1.0/+download/bzr-setup-1.0.0.exe">bzr</a> for windows to C:\Bazaar</p>
</li>
</ol>
<p>OK thats all the preparation steps. Launch C:\msys\msys.bat now to start a minimal unix-like shell. Now onto the meaty part;</p>
<ol>
<li>
<p>bzr branch http://www.gnome.org/~jstowers/bzr/jhbuild-win32</p>
</li>
<li>
<p>cp jhbuild-win32/win32-hacks/* . && mv profile .profile && mv perl.sh /bin/perl</p>
</li>
<li>
<p>edit pkg-config.sh to reflect the prefix you are going to install (or have installed) pkg-config to</p>
</li>
<li>
<p>edit jhbuild-win32/configs/gtk-win32.jhbuild so that full unix paths reflect the correct location of pkg-config.sh. Make sure /bin/perl and pkg-config.sh are executable</p>
</li>
<li>
<p>Close the shell and restart it so that it picks up your new .profile. Check that you can run all needed executables
perl --version && python --version && jhbuild --help && ./pkg-config.sh --version</p>
</li>
</ol>
<p>Oh, I seem to be forgetting one more thing.... The following command will build gtk+,atk,cairo,pango,zlib,libpng, and glib from source. No really, that is it [0].</p>
<ol>
<li>jhbuild -f jhbuild-win32/configs/gtk-win32.jhbuild build</li>
</ol>
<p><img alt="Built Gtk+ On Windows" src="http://www.johnstowers.co.nz/blog/static/images/imported/gtk-windows-works.jpg" /></p>
<h3>The Hacks (aka things I don't really understand)</h3>
<p>There are a few parts of this setup that still confuse me.</p>
<ul>
<li>
<p>/bin/perl is a thin wrapper around perl that quotes args and escapes path slashes. Its necessity is based around the interaction of the msys path escaping and what happens when autotools invokes glib-mkenums. It was based upon what I read <a href="http://www.glom.org/wiki/index.php?title=WindowsBuildInstructions">here</a>. It also writes a debug log to ~/perllog</p>
</li>
<li>
<p>~/pkg-config.sh is a wrapper around pkg-config.sh that quotes args. It is as described <a href="http://lists-archives.org/gtk-devel/05434-gtk-configure-can-t-find-cairo-pdf.html">here</a> but with a full prefix to pkg-config (so that you may use one that is installed in your jhbuild prefix). It also writes a debug log to ~/pkglog</p>
</li>
</ul>
<h3>Jhbuild Core Modifications</h3>
<p>I am currently hosting a <a href="http://www.gnome.org/~jstowers/bzr/jhbuild-win32">bzr branch </a>with my modifications to Jhbuild. I'm still cleaning some things up, but I hope that this is suitable for committing to SVN in the future. My modifications to Jhbuild are basically;</p>
<ul>
<li>
<p>Cleaning up all path handling code. That means using os.path.abspath and os.path.join internally. The msys does shell does some shiny things to convert unix paths to windows paths but it works best if the path it is passed is coherent and not a mix of both systems path separators.</p>
</li>
<li>
<p>Some path and environment variable fudging that checks all paths that get passed out to a shell, and converts them to unix format. Python on windows calls the windows CreateProcess function (that expects a windows executable). Therefor we starts a new msys/bin/sh.exe with the first arg the script we originally wished to start, followed by all the inferior args.</p>
</li>
<li>
<p>Added a <a href="http://bugzilla.gnome.org/show_bug.cgi?id=503679">binary module type</a>. This is basically like a tarball but instead of executing configure, make, make install you can define arbitary shell commands to execute post download and unpack. This is used to provide glib-bin and gtk-bin metapackages (which get binaries from<a href="http://www.gimp.org/~tml/gimp/win32/downloads.html"> Tor's gtk win32 site</a>). You can depend on the metapackages if you dont feel like compiling the gtk+/glib stack yourself and just want to build your app. You can also insert these binary modules in the dependency chain and use them to execute arbitrary scripts to fix (read: hack) some aspects of packages.</p>
</li>
<li>
<p>New Gtk+ and Glib modules files for binary installation, and source installation.</p>
</li>
<li>
<p>Use pythons build in functions to download files (instead of wget) and to untar/unzip archives (instead of tar and unzip command line programs). Note that this is not strictly necessary as there are builds of tar/unzip and wget for msy - but its easier from a developer perspective.</p>
</li>
</ul>
<h3>Remaining Work, Caveats (aka the full story)</h3>
<ul>
<li>
<p>libjpeg builds ok but doesn't seem able to generate a shared library. Hence libtiff fails and gtk builds without jpeg support.</p>
</li>
<li>
<p>glib-mkenums seems to insert arbiraty data (a directory listing) at the end of .h files. This results in many compiler warning about data after #endif.</p>
</li>
<li>
<p>Read the gtk-win32.jhbuildrc file.</p>
</li>
<li>
<p>Gtk doesn't set the correct paths to the pixbuf loaders and input modules. You will need to run (from the prefix to which Jhbuild installed everything)</p>
<ul>
<li>
<p>$JHBUILDPREFIX/bin/gtk-query-immodules-2.0.exe > $JHBUILDPREFIX/etc/gtk-2.0/gtk.immodules</p>
</li>
<li>
<p>$JHBUILDPREFIX/bin/gdk-pixbuf-query-loaders.exe > $JHBUILDPREFIX/etc/gtk-2.0/gdk-pixbuf.loaders</p>
</li>
</ul>
</li>
<li>
<p>You can change gtk to use the default theme by doing</p>
<ul>
<li>echo 'gtk-theme-name = "MS-Windows"' > $JHBUILDPREFIX/etc/gtk-2.0/gtkrc</li>
</ul>
</li>
<li>
<p>I have not tested this with SVN modules on windows, only tarball releases. I wasn't prepared to discover the compatibility of the msys provided autotools stack (yet).</p>
</li>
<li>
<p>--disable-docs on which/all modules?</p>
</li>
<li>
<p>Hookup to, and submit build reports to <a href="http://jhbuild.bxlug.be/">JhAutobuild</a></p>
</li>
<li>
<p>Python or msys or something buffers all the shell output. This means that there can be long periods (minutes) of nothing getting printed to the shell. If it looks like something has failed to build (no CPU/disk activity) the push 1,3 or 4 to retry, cancel, or drop into a shell respectively (i.e. the normal Jhbuild menu presented upon failure).</p>
</li>
</ul>
<h2>Part 2: I Just Want to Build my Application (not Gtk)</h2>
<p>It should be noted that it is not necessary to build Gtk (from source) if you just want to build a single application. Using the <a href="http://bugzilla.gnome.org/show_bug.cgi?id=503679">new binary module type</a>, I have prepared a gtk-binary metamodule that downloads all the needed files from <a href="http://www.gimp.org/~tml/gimp/win32/downloads.html">Tor's site</a>, and installs them for you. That sounds confusing, so lets take a randomly chosen example program [2]. Lets install <a href="http://mooedit.sourceforge.net/">medit</a>, from source, using Jhbuild.</p>
<p>Starting with the example configs/gtk-win32.jhbuild file change the module and modulesets lines to</p>
<div class="codehilite"><pre><span></span><span class="n">moduleset</span> = [
<span class="n">os</span>.<span class="n">path</span>.<span class="nb">join</span>(<span class="n">thisdir</span>,<span class="s">'gtk-win32-bin.modules'</span>),
<span class="n">os</span>.<span class="n">path</span>.<span class="nb">join</span>(<span class="n">thisdir</span>,<span class="s">'meddit.modules'</span>)
]
<span class="n">modules</span> = [<span class="s">'meddit'</span>]
</pre></div>
<p>Add a module_autogen line for meddit</p>
<div class="codehilite"><pre><span></span><span class="c c-Singleline">#Module specific configure arguments passed in addition to autogenargs</span>
<span class="n">module_autogenargs</span>[<span class="s">'meddit'</span>] = <span class="s">' --without-xml --disable-help --with-python=no'</span>
</pre></div>
<p>And the final step....</p>
<div class="codehilite"><pre><span></span>jhbuild -f jhbuild-win32/configs/gtk-win32.jhbuild build
</pre></div>
<p><img alt="Meddit running on windows" src="http://www.johnstowers.co.nz/blog/static/images/imported/meditsmall.jpg" /></p>
<h3>Conclusion</h3>
<p>I got into this because I wanted to be able to build pygoocanvas on windows, so I could hack on Conduit there. I still have not done that, instead I have;</p>
<ul>
<li>
<p>gotten royally distracted from the original goal,</p>
</li>
<li>
<p>hacked on Jhbuild,</p>
</li>
<li>
<p>learned a lot about msys/mingw and autotools on windows,</p>
</li>
<li>
<p>hopefully been able to contribute something useful back to the community that makes it easy for joe normal and jane developer to test their app on windows</p>
</li>
<li>
<p>Oh and its not yet finished (but I have tested it a few times on fresh installs, and it should just work) yet</p>
</li>
<li>
<p>Pretty normal summary for a FOSS hackers week, correct?</p>
</li>
</ul>
<p>I would really like help from some people in testing this further, so go ahead, test it today. Ive made it 12 easy steps for you to follow.......</p>
<h3>References (thanks)</h3>
<ul>
<li>
<p><a href="http://tech.groups.yahoo.com/group/gimpwin-users/message/16330">Building Gtk+/Gimp on Win32 (May 2005, Tor Lillqvist)</a></p>
</li>
<li>
<p><a href="http://www.go-evolution.org/Building_Evolution_on_Windows">Building Evolution on Windows (Feb 2007)</a></p>
</li>
<li>
<p><a href="http://mail.gnome.org/archives/gtk-devel-list/2005-January/msg00091.html">An attempt at Win32 build instructions (Jan 2005, Owen Taylor)</a></p>
</li>
<li>
<p><a href="http://www.glom.org/wiki/index.php?title=WindowsBuildInstructions">Compiling Glom on Windows (Dec 2007, Armin Burgmeier)</a></p>
</li>
<li>
<p><a href="http://lists-archives.org/gtk-devel/05434-gtk-configure-can-t-find-cairo-pdf.html">pkg-config.sh (Jun 2007, Cody Russel)</a></p>
</li>
<li>
<p><a href="http://osdir.com/ml/gnu.mingw.user/2002-11/msg00298.html">Circular deps of pkg-config (Tor Lillqvist)</a></p>
</li>
</ul>
<p>[0] A full Gtk+ build from source takes about 6 hours on Windows XP (running in VMware getting both 2.4Ghz Pentium 4 cores)</p>
<p>[1] Ok, not the complete stack. To get around some circular dependency issues I start with binary versions of pkg-config iconv and gettext. With pkg-config present I then build Glib from source. I also use a binary version of poppler because it depends on Gdk while Gtk depends on poppler.</p>
<p>[2] Well not entirely random. It was already advertised as being able to builad on windows. Also it failed during make install, complaining about missing share/mime/packages/freedesktop.org.xml and bin/update-mime-database. I created both as empty files and it was successfully installed.</p>Jhbuild and Windows - We Meet Again2007-12-13T01:56:18+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-12-13:2007/12/13/jhbuild-and-windows-we-meet-again/<h3>The Good News</h3>
<p>Following on from <a href="http://www.johnstowers.co.nz/blog/index.php/2007/12/11/jhbuild-adventures-on-windows/">my last post</a> I have continued work on getting Jhbuild to run on windows (within msys). The basic problem which I identified in my last post was the struggle between Python (for Windows) expectations about paths, and the hoops that msys and cygwin jump through to allow unix style paths to be used. Basically os.path.join and friends do the correct thing providing the subsequent path is used within python alone. Things get all difficult when that path is then passed out to a configure script, or as an argument to a shell command. With that in mind however, a picture speaks a thousand words....</p>
<p><img alt="JHBuild on Windows" src="http://www.johnstowers.co.nz/blog/static/images/imported/jhbuildwindowscrop.png" /></p>
<ol>
<li>
<p><strong>It works...</strong></p>
</li>
<li>
<p><strong>On windows...</strong></p>
</li>
<li>
<p><strong>Through magic....</strong></p>
</li>
</ol>
<h3>The Status</h3>
<p>Jhbuild is successfully able to build a test moduleset, and even some real modules such as zlib and iconv on windows using the mingw compiler. Things are looking promising and I will continue to play with this tomorrow. Its a bit of a pain to debug because of the inconsistancies between how subprocess.Popen behaves on windows and on linux. So far my changes can be summarised as follows</p>
<ol>
<li>
<p>Add Makefile.win32 and patch install-check to just call /usr/bin/install</p>
</li>
<li>
<p>Replace wget/curl with pythons built in urllib.urlretrieve</p>
</li>
<li>
<p>Replace tar/unzip with pythons built in tarfile and zipfile modules</p>
</li>
<li>
<p>Misc fixes to get Jhbuild to start including disabling the tray icon etc</p>
</li>
<li>
<p>Refactor buildscript.execute (the subprocess.Popen) wrapper to clean it up and to special case local script commands from system commands (because of differences in how win32 treats executing executables in the current dir). Best summarised as the following</p>
<p>def execute_script(self, command, args_str='', cwd=None, extra_env=None):
# THIS IS THE MAIN WIN32 HACK
if sys.platform.startswith('win') and command[0].startswith('./'):
command = ['sh', command[0].replace('./','')] + command[1:]</p>
</li>
</ol>
<p>Jhbuild still works on Linux just as it used to (more or less, I need to port over some more things to the new buildscript.execute() functions). Wget/tar/curl/unzip etc are still used on Linux. In general the changes so far have been self contained and should not break regular operation</p>
<h3>The Install Procedure</h3>
<p>How can you try this pre-alpha code out?. Follow these steps on windows and check out from my bzr branch (or use the <a href="http://bugzilla.gnome.org/show_bug.cgi?id=502970">patch against SVN</a>)</p>
<ol>
<li>
<p>Install MinGW combined installer (MinGW-5.1.3) to C:\mingw</p>
</li>
<li>
<p>Install msysCORE-1.0.11-2007.01.11 to C:\msys</p>
</li>
<li>
<p>Run C:\msys\postinstall\pi.bat, answer yes when it asks whether mingw is already installed and point it to C:/mingw. Launching C:\msys\msys.bat now provides us with a minimal unix-like shell.</p>
</li>
<li>
<p>Install Python 2.5.1 to default location (C:\Python25)</p>
</li>
<li>
<p>Install bzr for windows to the (C:\Bazaar)</p>
</li>
<li>
<p>Start msys.bat (in C:\msys)</p>
</li>
<li>
<p>Install Jhbuild</p>
<ol>
<li>
<p>bzr co http://www.gnome.org/~jstowers/bzr/jhbuild-win32 jhbuild-win32</p>
</li>
<li>
<p>cd jhbuild-win32</p>
</li>
<li>
<p>make -f Makefile.win32 install</p>
</li>
</ol>
</li>
<li>
<p>Add python to path. Other path manipulation is done within JHBuild. Add the following line to ~/.profile</p>
<p>export PATH=$PATH:/c/Python25:/c/Python25/bin</p>
</li>
</ol>
<p>You should now have a minimal msys install with JHBUILD. Test by performing</p>
<ul>
<li>
<p>cd ~/bin</p>
</li>
<li>
<p>python jhbuild --help</p>
</li>
</ul>
<p>Finally, you can now install the test modules with</p>
<div class="codehilite"><pre><span></span>python jhbuild -f jhbuild-win32/test.jhbuildrc build test
</pre></div>
<p>Standard disclaimers apply, alpha quality code, help appreciated, yada yada yada. Now back to Conduit hacking....</p>Jhbuild Adventures on Windows2007-12-11T19:20:18+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-12-11:2007/12/11/jhbuild-adventures-on-windows/<p><strong>Background:</strong>
I'm going home to my parents house for Christmas, and because I don't have a laptop, I thought it would be the perfect opportunity to use their computer to get <a href="http://www.conduit-project.org/">Conduit</a> working on windows. However I could not find windows builds of (<a href="http://developer.berlios.de/projects/pygoocanvas/">py</a>)<a href="http://sourceforge.net/projects/goocanvas">goocanvas</a> anywhere. Never minding a challenge I thought it would be a good experiment to see if I could build pygoocanvas for windows via cross-compilation or natively - both using <a href="http://live.gnome.org/Jhbuild">jhbuild</a>.</p>
<p>I like to think that F/OSS works because people keep re-inventing the wheel in new and obscure ways everytime they do something, so what follows is my rambling tale of cross-compiling-gtk-for-windows wheel reinvention.</p>
<h2>Cross compiling Gtk+ for Windows using Jhbuild</h2>
<p>Starting with the <a href="http://live.gnome.org/Cross_compiling_GTK+_for_Win32">work of Alberto Ruiz</a> and <a href="http://mail.gnome.org/archives/gtk-devel-list/2007-June/msg00293.html">Piotr Gaczkowski </a>I thought I would setup a jhbuild environment to cross compile the Gtk+ stack. After a bit of fiddling I managed to get everything to build fine using mingw32 and jhbuild. The following files work successfully on Ubuntu gutsy and can build the latest stable releases of Gtk+/cairo/glib/libglade.</p>
<ul>
<li>
<p><a href="https://svn.greenbirdsystems.com/gtk-win32/trunk/gtk+-win32-john.jhbuildrc">gtk+-win32-john.jhbuildrc</a></p>
</li>
<li>
<p><a href="https://svn.greenbirdsystems.com/gtk-win32/trunk/gtk+-win32-john.moduleset">gtk+-win32-john.moduleset</a></p>
</li>
</ul>
<p>At this point I moved onto trying to build pygobject. This is where things got difficult. I tried a few different ways to get the cross compilation environment to see python. With some inspiration from <a href="http://www.glom.org/wiki/index.php?title=WindowsBuildInstructions">this page on compiling glom for windows,</a> I copied across the entire Python25 directory from a windows box and added it to the appropriate LDFLAGS and CFLAGS in jhbuild. This allowed pygobject to pass configure. However it now failed to build;</p>
<div class="codehilite"><pre><span></span>In file included from /usr/include/python2.5/Python.h:57,
from pygobject.h:5,
from pygobject-private.h:9,
from gobjectmodule.c:27:
/usr/include/python2.5/pyport.h:246:24: sys/select.h: No such file or directory
</pre></div>
<p>I went back to the windows machine and checked to see if sys/select.h was present in mingw. No luck. OK at this point I was stuck. Compiling wasn't looking likely, and I still wasnt even confident that if it did compile it would manange to actually link and run (due to different compiler versions and platforms, etc). I googled for a bit and saw that its now <a href="http://boodebr.org/main/python/build-windows-extensions">possible to build python extensions using mingw32</a> and that people had managed to <a href="http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/">cross compile python using mingw</a>. Hmm, I pondered..... 'lets cross compile Python25 using mingw and then build pygobject using that.' Furthur googling revealed <a href="http://bugs.python.org/issue1597850">this python bug+patch</a> for cross compiling using mingw. This is going to be easy.......</p>
<p>I added a new section to my modules file that would apply the patch to the Python 2.5.1 before building it. <strong>YOU FAIL AGAIN JOHN</strong>.</p>
<div class="codehilite"><pre><span></span>checking for %zd printf() format support... configure: error: cannot run test program while cross compiling
</pre></div>
<p>Hmm. I tried several strategies to get around this (including supplying a config.cache file with ac_cv_printf_zd_format=yes) but with no luck. You can see my plea for help on the bug report.......</p>
<p>At this point an entire day of my life had been irrevocably taken from me. I retired to bed. I would not let windows beat me....</p>
<h2>Native compilation of Gtk+ for Windows using Jhbuild</h2>
<p>OK. At this point I thought I might as well start again, but compile natively on windows using <a href="http://www.mingw.org">mingw and msys</a>. In hindsight it probbably would have been easier to follow most of the <a href="http://www.glom.org/wiki/index.php?title=WindowsBuildInstructions">instructions for glom</a> verbaim, but I decided to continue my quest at wheel re-invention.</p>
<p>I <a href="http://bugzilla.gnome.org/show_bug.cgi?id=499497">read that Alberto</a> was thinking about ways to make Jhbuild work on windows - a good goal. Starting with his patch on that bug report I <a href="http://bugzilla.gnome.org/attachment.cgi?id=100733&action=edit">modified it</a> slightly so that tar, gunzip and friends wouldn't need to be installed. Furthur to that I set about <a href="http://bugzilla.gnome.org/show_bug.cgi?id=502970">hacking up some patches</a> to allow jhbuild to at least run on windows. A few hours later and I was sure that this was going to take a few more hours :-). I now had a working (ish) minimal easy to set up windows build environment according to <a href="https://svn.greenbirdsystems.com/gtk-win32/trunk/mingw-setup.txt">these instructions</a>..... or so I thought.</p>
<p>Jhbuild managed to successfully parse my minimal jhbuildrc and module files, download and untar iconv but then it failed. Basically there is considerable work and thought needed to see how this should operate in msys/cygwin. The problem is already explained in part <a href="http://www.glom.org/wiki/index.php?title=WindowsBuildInstructions">here</a>, but I will summarise.</p>
<p>Python's os.path and friends do a good job at converting paths to the native system format. Unfortunately the msys and cygwin shells also convert paths. For instance msys automagically converts /foo/bar to C:\msys\foo\bar. Basically it means that python converts paths to windows format, but msys likes them in unix format (because its a unix shell environment). This results in files not getting found, and gross composite paths like --prefix=C:\foo\bar/baz/bob getting passed to configure.</p>
<p>So, I had a bit of a look at how this could be solved in Jhbuild. I guess there are a few strategies</p>
<ol>
<li>
<p>Create a whole bunch of utility functions join_path(), prepend_path(), etc. And call these instead of the system ones and only escape to windows format <em>if you are running in windows</em>, and not <em>running in windows just kidding you are really running in cygwin haha</em>.</p>
</li>
<li>
<p>Remove all os.path and friends calls. Be really naive and hard code all path operations to use unix semantics. Then explictly note that Jhbuild on windows must be run from within a unix shell like msys/bash or cygwin/bash.</p>
</li>
<li>
<p>Magic. Come up with some sort of paranormal event that would <a href="http://www.randi.org/">win GNOME $1 million dollars</a> which could then pay a developer to fix all this for me.</p>
</li>
</ol>
<h2>Conclusion</h2>
<p>Well things didn't work out as well as I hoped. I will probbably go and compile everything <a href="http://www.glom.org/wiki/index.php?title=WindowsBuildInstructions">without jhbuild</a> and be happy. I did learn that;</p>
<ul>
<li>
<p>Jhbuild on windows is a worthwhile cause. It would really help in <a href="http://jhbuild.bxlug.be">automatically (jhautobuild)</a> testing things like glib and GIO (at least giving devs a compile yes/no signal)</p>
</li>
<li>
<p>It could make generating the latest tarballs really easy, perhaps even sharing the same modulesets as those maintained by the release team</p>
</li>
<li>
<p>For extra bling it could even automatically create installers by tracking installed files using a strategy similar to checkinstall.</p>
</li>
</ul>
<p>Help, thoughts or advice appreciated. For all the luck I have had the last few days I would not be surprised if I have missed something fundamentally obvious to this whole process. Heck if some fairy was to drop a pygoocanvas installer for windows in my mail box then I would be super happy. Oh and did I mention that im really looking forward to gio!</p>The Big Move2007-12-06T13:37:03+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-12-06:2007/12/06/the-big-move/<p>The short news is that <a href="http://www.conduit-project.org/">Conduit</a> now <a href="http://svn.gnome.org/viewvc/conduit/">lives in GNOME SVN</a>. The move took a little longer than I would have hoped, but thats largely my fault. I would like to express my thanks to <a href="http://blogs.gnome.org/ovitters">Olav Vitters</a> for importing the SVN repository, his excellent response time to my emails, and the awesome <a href="http://mango.gnome.org">mango</a> system he created.</p>
<h3>Whats Good</h3>
<p>Conduit trunk is now pretty much feature frozen. It feels really good to finally cross off the TODO list and fix some of the bugs that I created (due to some poor choices when I started the project). While I will blog again soon about the shiny <a href="http://www.conduit-project.org/wiki/0.3.5">new features</a> for the moment I will talk about one useful piece of code that I am proud of, and that may be useful to others.</p>
<p>Initially I used an in-memory python dictionary based database to store the object mappings which represent a sync relationship between two pieces of data (the mapping DB). I pickled this to disk when conduit closed. Unfortunately this was not only horribly gross (memory usage was huge), it was embarrassingly short sighted, as the one place I care most about types is this code path. We use dictionaries and object hashes extensively in the sync logic and its really hard to debug sync problems when you (for example) don't notice that a dataprovider returns the wrong type until you unpickle it a few days later.</p>
<p>To remedy this I needed (wanted) to use a database that had few or no external dependencies, that was a bit tougher on type safety, and that was usable from multiple threads. After a while I decided that because sqlite was now part of python2.5 this would be a good choice, and that I could get around the threading issues some other way.</p>
<p>Not wanting to re-invent the wheel completely I started with a few existing pieces of code</p>
<ul>
<li>
<p>Christian Hergert had <a href="http://vwdude.com/blog/?p=9">written</a> a nice Gtk.TreeModel wrapper around a sqlite DB
<a href="http://vwdude.com/dropbox/pystore/">http://vwdude.com/dropbox/pystore/</a></p>
</li>
<li>
<p>Louis RIVIERE had written a threadsafe wrapper around sqlite that allows the DB to be accessed from more than one thread
<a href="http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/526618">http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/526618</a></p>
</li>
</ul>
<p>I glued these together and created a really nifty (yet another) DB abstraction layer that can be used in either single or multiple threaded applications, and then displayed in a Gtk.TreeView with NO additional effort. Whats more, the DB also has a LRU cache that dramatically reduces the number of calls into the DB when displayed in a Gtk.TreeView. For more information check out the following</p>
<ul>
<li>
<p><a href="http://svn.gnome.org/viewvc/conduit/trunk/conduit/Database.py?view=log">The Database abstraction (single and multiple thread compatible)</a></p>
</li>
<li>
<p><a href="http://svn.gnome.org/viewvc/conduit/trunk/conduit/gtkui/Database.py?view=markup">The Gtk.TreeModel wrapper</a></p>
</li>
<li>
<p><a href="http://svn.gnome.org/viewvc/conduit/trunk/tools/test-db-wrapper/main.py?view=markup">An Example of how to use all these parts together
<img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/screenshot-mainpy.png" />
</a></p>
</li>
</ul>
<p>All of this has been really heavily tested and everything seems to work as expected*........ except for the LRU cache, which I broke somewhere - hmm. I should really fix that.</p>Another n810 Post...2007-11-10T12:58:49+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-11-10:2007/11/10/another-n810-post/<p>Thanks to the awesome work of Thomas Van Machelen, Conduit has the <a href="http://www.conduit-project.org/browser/trunk/conduit/hildonui">beginnings</a> of a port to the hildon ui. Whats more impressive, is that unlike me (who owns a n800), Thomas has been doing all this in an emulator. So anyone who reads this, consider this an endorsement of Thomas request for a developer discount code!</p>
<p><img alt="Conduit on n800" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-n800.jpg" /></p>
<p>Conduit on the n8xx is something I have been looking forward to for a while. The first and most tangible benefits will be giving n8xx users the ability to use Conduits photo sync facilities to upload pictures to all the webservices we support. Beyond this however, there is much more to offer;</p>
<ul>
<li>
<p>PIM sync using the evolution-python bindings on both the host PC and the n8xx</p>
</li>
<li>
<p>File sync to the n8xx without having to plug it into USB and transfer things via the memory card. By using conduits built in network sync (over python XMLRPC), sync is as easy as letting the PC and n8xx be on the same network.</p>
</li>
</ul>One Thousand2007-10-31T18:06:55+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-10-31:2007/10/31/one-thousand/<p>Conduit recently reached 1000 commits, and with it came together some functionality that has been brewing for a while.</p>
<ul>
<li>
<p>Far far far improved nokia n800 support</p>
<ul>
<li>
<p>You can now sync files, photos, music, and videos to your nokia n800 in an intelligent way (i.e. its not just a dumb file/folder sync).</p>
</li>
<li>
<p>The ability to transcode said data to the most appropriate format for the device. <a href="http://www.conduit-project.org/browser/trunk/conduit/modules/AudioVideoConverterModule.py">FFmpeg/Mencoder is used to transcode audio/video</a>, and gtk.Pixbuf is used to <a href="http://www.conduit-project.org/browser/trunk/conduit/modules/PhotoConverterModule.py">convert photos</a>. The converters use conduits built in dynamic modular plugin system, so if someone wants to write a audio video converter using GStreamer then please contact me.</p>
</li>
<li>
<p>Also, those that are familiar with FFmpeg/Mencoder/Transcoding please feel free to contribute preset encoding profiles (video codec, size, etc). Presets are stored <a href="http://www.conduit-project.org/browser/trunk/conduit/datatypes/Video.py#L4">here (video)</a>, <a href="http://www.conduit-project.org/browser/trunk/conduit/datatypes/Audio.py#L4">here (audio)</a> and <a href="http://www.conduit-project.org/browser/trunk/conduit/datatypes/Photo.py#L6">here (photo).</a></p>
</li>
</ul>
</li>
</ul>
<p><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/n800-conduit.jpg" /></p>
<p><em>Another one of those conduit screenshots that always look the same</em></p>
<ul>
<li>
<p>Looser dependence on gtk. This means that conduit can be run from a console (-c command line switch) without gtk/goocanvas installed. This reduces the memory usage by about 10Mb!</p>
</li>
<li>
<p>Massive refactoring effort to aid more <a href="http://www.conduit-project.org/browser/trunk/conduit/modules">code sharing between dataproviders</a> and the complete <a href="http://www.conduit-project.org/browser/trunk/conduit/gtkui">decoupling of the GUI from the core application</a>. This allows other guis to written very independently of the rest of the application <em>[1]</em>. Stay tuned for a native hildon gui for nokian800 and/or ubuntu mobile.</p>
</li>
<li>
<p>Rhythmbox support. A simple dataprovider has been added that is able to sync rhythmbox static playlists with things. Initially this is used to test sync music with n800 (with the ability to transcode audio formats), but in my ideal world, sync with rhythmbox would involve</p>
<ul>
<li>Improving rhythmboxes DBus interface to support adding/removing songs to the library and querying playlists (including smart playlists).</li>
</ul>
</li>
<li>
<p>Remember because we are gnomevfs aware some cool things are possible. For instance, one can sync + transcode videos/music to/from their phone which is mounted using gnomevfs-obex.</p>
</li>
<li>
<p>Work is currently proceeding furiously on</p>
<ul>
<li>
<p>Integrating with opensync</p>
</li>
<li>
<p>Removing the twisted dependence, and using pythons built in xml-rpc server for network synchronization</p>
</li>
<li>
<p>Replacing our current in memory database with one based on sqlite</p>
</li>
<li>
<p>Support for iPod photos</p>
</li>
<li>
<p>Test coverage andbug fixing.</p>
</li>
</ul>
</li>
</ul>
<p>Anyway, svn HEAD is pretty rocky at the moment, but please experiment. Once the database changeover has taken place things should stabilise for a 0.3.5 release. This will be a release candidate and then we will focus on bug fixing only.</p>
<p>Ideally we will release a stable 0.4.0 release at or before <a href="http://live.gnome.org/Melbourne2008">gnome.conf.au</a>. As an aside, I would be interested in talking to any of the RH online desktop folks at <a href="http://linux.conf.au/">linux.conf.au</a> as I am unable to make it to any of the northern hemisphere conferences on account of me being poor and in New Zealand.</p>
<p><em>[1] Excluding a dependence on gnomevfs (yuck). Hopefully gio/gvfs will make all my dreams come true.</em></p>Google Online Desktop2007-10-10T12:53:20+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-10-10:2007/10/10/google-online-desktop/<p>Stumbled upon <a href="http://googlesystem.blogspot.com/2007/10/google-online-desktop.html">this</a>. Discuss..</p>
<p><a href="http://bp3.blogger.com/_ZaGO7GjCqAI/RwoHrDsvOFI/AAAAAAAAFYw/0EAmWSqGHyA/s1600-h/googleedesktop.jpg"><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/googleedesktop-small.png" /></a></p>Unintended Uses2007-10-06T00:11:04+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-10-06:2007/10/06/unintended-uses/<p>I just returned to my computer to find that nautilus crashed somewhere in the midst of copying my 50GB (1000 odd holiday videos) from my external hard disk to my <a href="http://www.serverelements.com/naslite-2-usb.php">NAS</a> via FTP. Not wanting to work out where exactly the crash happened and what files were incomplete or still needing to be copied I thought 'hey, I could use <a href="http://www.conduit-project.org/">Conduit</a> to do a one way sync'. So I did, and it worked.</p>
<p><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/terminalsync.jpg" /></p>
<p>Its always cool when software you have written manages to scratch an itch that you didn't design it for!</p>Conduit 0.3.4 - 37% more integrated2007-10-03T01:21:57+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-10-03:2007/10/03/conduit-034-37-more-integrated/<p>Wahoo!, after mad busy weeks of development, I just released <a href="http://www.conduit-project.org/wiki/0.3.4">Conduit 0.3.4</a>. Check out the <a href="http://video.google.com/videoplay?docid=-262844619875208739&hl=en">screencast</a> (<a href="http://www.box.net/shared/47kpxl2ykg">download here</a>). This release is fresh with new features, and heaps of polish and integration work. These new features include;</p>
<ul>
<li>
<p>Google calendar support (Paul Novotny).</p>
</li>
<li>
<p>Youtube dataprovider added (Renato Araujo).</p>
</li>
<li>
<p>Photo sites are now two way, you data is no longer locked with one provider. Also see <a href="http://bugzilla.gnome.org/show_bug.cgi?id=480754">#480754</a> for improvements to Fspots DBus interface allowing two way photo sync with FSpot.</p>
</li>
<li>
<p>Nautilus extension. (disabled by default).</p>
</li>
<li>
<p>EOG plugin added. (disabled by default).</p>
</li>
<li>
<p>Conduit now supports resizing and converting photos before uploading them to photo sites. This infrastructure means improved support for other types of conversions, such as transcoding music and video, will be added in the future.</p>
</li>
<li>
<p>Progress indicator shows percent complete during synchronization.</p>
</li>
<li>
<p>Add support for Facebook photo upload.</p>
</li>
<li>
<p>Add included web browser for authenticating against web sites.</p>
</li>
</ul>
<p><strong>Website Authentication</strong></p>
<p>The last point deserves some explanation. Currently, to authenticate against some sites the user must log in either once (Flickr), or once per session (box.net). This is difficult when using the system default web browser, if the browser is currently running on a different virtual desktop to conduit, the user often doesnt notice the web browser go to the login website. This is compounded by the different APIs of some sites. Flickr is good, I can just call <em>am_i_authenticated(token)</em> in a loop, and wait for success, or time out. Facebook is not good. Any call involving the facebook token before it has been validated (i.e. user logs in) invalidates the pending login. Because there is no guarenteed way to block in all cases (i.e. until the user logs in and closes the window), things like gnome.open_url() or python.webbrowser() are unsuitable. This leaves me with two options. I can wait a random amount of time, and hope its long enough for the user to log in, or I can write my own web browser that will block until the user closes it. Suggestions for option 3 are appreciated....</p>
<p><a href="http://www.conduit-project.org/wiki/Screenshots"><img alt="" src="http://www.johnstowers.co.nz/blog/static/images/imported/weblogin.jpg" /></a></p>
<p>I know this is not ideal. I hope something like <a href="http://live.gnome.org/OnlineDesktop/Components">web-login-browser</a> can solve my needs. Also, I know my gtkmozembed based browser has some (crasher..) issues (maybe related to <a href="http://cgwalters.livejournal.com/7453.html">this</a>). I hope to fix them and/or move to a <a href="http://live.gnome.org/PyWebKitGtk">pywebkitgtk</a> based browser, depending on what GNOME favors.</p>
<p>Here is a <a href="http://video.google.com/videoplay?docid=-262844619875208739&hl=en">screencast</a> showing the latest version and its features. You may <a href="http://www.box.net/shared/47kpxl2ykg">download the original here</a>. Apologies for the google video quality.</p>
<p>[googlevideo]http://video.google.com/videoplay?docid=-262844619875208739&hl=en[/googlevideo]</p>
<p><strong>Boston Summit</strong></p>
<p>Unfortunately, owing to my proximity to the opposite side of the world as Boston, I will not be attending the <a href="http://live.gnome.org/Boston2007">summit</a>. I understand that the GNOME online desktop is on the agenda, and will no doubt be discussed at some length. I hope those involved in the discussion try out conduit before the summit, as it would be a shame if the future of the GNOME online desktop was set without due consideration. This is not a criticism, sour grapes, or anything like that. Its just a reflection of what appeared to happen after GUADEC - which I also did not attend (New Zealand = isolated and distant).</p>
<p>Have fun with Conduit. Please <a href="http://www.conduit-project.org/newticket">report bugs</a>, join the <a href="http://mail.gnome.org/mailman/listinfo/conduit-list">mailing list</a>, and abuse me at will. We are currently moving to GNOME servers, so if my server goes down from all the traffic I promise it will be the last time.</p>What Ever Happened To?2007-09-25T23:34:04+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-09-25:2007/09/25/what-ever-happened-to/<p>There are a number of lively discussions currently going on in <a href="http://mail.gnome.org/archives/desktop-devel-list/">desktop-devel</a> with regard to new module proposals <em>[1]</em>. This reminded me of a few projects I used to follow that I have not heard from in a while. Hoping not to offend any of the developers I therefore ask,</p>
<p><strong>What ever happened to?</strong></p>
<ul>
<li>
<p>Gnome Formatter
A HAL pretty formatting utility, so I may finally easily format my USB keys. Description <a href="http://blog.paulbetts.org/index.php/2007/01/29/call-for-developers-gnome-format/">here</a>, and <a href="http://www.barisione.org/blog.html?p=65">here
</a><img alt="" src="http://farm1.static.flickr.com/160/387928386_b69d1d6184_m.jpg" /></p>
</li>
<li>
<p><a href="http://live.gnome.org/GSmartMix">GSmartMix</a> + <a href="http://www.pulseaudio.org/">PulseAudio</a> + <a href="https://tango.0pointer.de/mailman/listinfo/libsydney-discuss">LibSydney</a>
A dream bunch of projects providing policy for sound devices. GMartMix lives between gstreamer and pulseaudio (i hope...) to allow applications to mute, pause, and have per application volume control, although the latter may be the domain of Pulseaudio. LibSydney said to replace esd in GNOME (finally, down with cruft).</p>
</li>
<li>
<p><a href="http://tw.apinc.org/weblog/tag/mathusalem">Mathusalem: Long running task manager</a>
This SOC2006 project built sufficient buzz that KDE has now acquired an <a href="http://dot.kde.org/1169588301/">equivalent</a> in KDE4. With Nautilus currently being broken to integrate the great new gvfs work, perhaps this would be a good time to play</p>
</li>
<li>
<p><a href="http://blogs.gnome.org/desrt/2007/02/18/panel-composite-bin/">New gnome-panel bling</a>
Ohhh shiny. Im happy to see this come up on ddl, users like shiny things. Some more gnome-panel <a href="http://live.gnome.org/GnomePanel/Future">ideas here</a>.</p>
</li>
<li>
<p>Libslab, <a href="http://live.gnome.org/GUADEC2006/PrintedContent/CoreAbst">Tiles</a> and the New UI Metaphor, including <a href="http://guadec.org/node/635">Gnome Main Menu.</a></p>
</li>
<li>
<p>Gnome-control-center single window mode.
I know it met with resistance when it was introduced but I quite liked it!
<img alt="Gnome Control Center Shell" src="http://www.johnstowers.co.nz/blog/static/images/imported/gnome-ccontrol-center-00-01.jpg" /></p>
</li>
<li>
<p><a href="http://log.emmanuelebassi.net/archives/2007/03/of-angels-and-angles/">GtkApplication and GtkUnique</a>
Perhaps GtkApplication is a smart place to put the <a href="http://www.kryogenix.org/days/2007/09/25/1440">window DND stuff</a> (<a href="http://bugzilla.gnome.org/show_bug.cgi?id=321211">now that nautilus supports XDS</a>)</p>
</li>
<li>
<p><a href="http://live.gnome.org/SessionManagement/NewGnomeSession">New GNOME Session Code</a>
TOPAZ type infrastructure which may <a href="http://www.gnome-look.org/content/show.php?content=44516">allow saving tasks to a file</a>, or even a <a href="http://gabriel.hurley.googlepages.com/userinterface">project or task oriented desktop</a>.</p>
</li>
<li>
<p><a href="http://www.gnome.org/~davyd/gnome-2-14/">Metacity Compositor</a>
Kwin is shipping with a compositor, as is the xfce window manager. It would be a shame to have nothing comparable (yes I am aware of Compiz, and sometimes it hurts to use)</p>
</li>
</ul>
<p>Anyway, its great to see the action on ddl, and the excitement surrounding GNOME again. I look forward to the next release. I will be releasing a Conduit 0.3.9 (the RC before 0.4.0) this weekend, and here is a hint about my next blog post: <em>Conduit as a blessed GNOME dependancy, how can we integrate Sync into the desktop?</em></p>
<p><em>[1] I am also hoping that <a href="http://www.gnome.org/projects/tracker/">tracker</a> and/or <a href="http://wiki.freedesktop.org/wiki/XesamAbout">xesam</a> gets proposed this cycle.</em></p>Expanding the Sync Space2007-09-03T22:55:17+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-09-03:2007/09/03/expanding-the-sync-space/<p>No screenshots this time, but still good news. <a href="http://www.conduit-project.org/">Conduit</a> tries to be a good citizen, integrating itself with GNOME technologies. Recently the following hacking has been going on (not all items are complete)</p>
<ul>
<li>
<p>A Eye Of Gnome plugin allowing photo upload/sync to Flickr, Picasa, SmugMug, Facebook (soon)</p>
</li>
<li>
<p>Gnome user documentation</p>
</li>
<li>
<p>Themable icons</p>
</li>
<li>
<p>Patches to improve FSpots DBus interface to allow two way syncs with photo sites</p>
</li>
<li>
<p>Continued work on OpenSync integration. We have limited support for arbitary opensync plugins!</p>
</li>
<li>
<p>Canola, n800, Rhythmbox and general music synchronization</p>
</li>
</ul>
<p>There have also been some new contributions</p>
<ul>
<li>
<p>Two way support for Flickr, Picasa and Smugmug. Now you can move your photos between sites (AKA <a href="http://www.movemydata.org/">move my data</a>)</p>
</li>
<li>
<p>A youtube dataprovider (sync your fave videos each morning)</p>
</li>
<li>
<p>Google calendar support (sync evolution with google calendar, iPod, etc)</p>
</li>
</ul>
<p>Anyway if you are looking to get involved with Conduit now is a great time. I have just updated the docs on <a href="http://www.conduit-project.org/wiki/WritingADataProvider">how to write a dataprovider</a>. Here are some ideas for dataproviders people may wish to write (hint hint)</p>
<ul>
<li>
<p><a href="http://www.conduit-project.org/ticket/64">Support Amazon S3</a></p>
</li>
<li>
<p>Support thunderbird contacts (mork library <a href="http://tarvid.kallimagarden.com/mork/">here</a>)</p>
</li>
<li>
<p>Support Liferea 1.4 feeds (they now are stored in a sqlite database - easy to parse)</p>
</li>
<li>
<p><a href="http://www.conduit-project.org/ticket/18">Delicious support</a></p>
</li>
</ul>
<p>There will be some new features landing in trunk soon that will allow arbitrary parameters during the conversion process, i.e. transcoding audio/video to another format, resizing photos, compressing files, etc. Watch this space, <a href="http://www.conduit-project.org/wiki/Build">download conduit</a>, and go tell people it rocks!</p>Autotools Bribe With Beer2007-08-20T23:33:44+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-08-20:2007/08/20/autotools-bribe-with-beer/<p>I have a love-hate relationship with autotools. I love the thought of a parallel universe where it doesn't exist, and hate it with the fiery passion of 1000 suns the rest of the time. I just released <a href="http://www.conduit-project.org/wiki/0.3.3">Conduit 0.3.3</a> and have had some reports of some build errors. The recurring build error is basically something like</p>
<p>Running intltoolize…
Running aclocal-1.4…
aclocal: configure.ac: 18: macro `AM_PATH_PYTHON’ not found in library</p>
<p>The problem is, by some combination of luck and ignorance I have never been able to reproduce this or fix it (on Ubuntu). The additional confusion is that Conduit is python, so I understand even less of the interaction of autotools. I therefor have an offer;</p>
<p><strong>I will happily pay the person who fixes the <a href="http://www.conduit-project.org/browser/trunk/configure.ac">horrible autotools setup in Conduit</a> a crisp $NZ50. This cool cash can be redeemed from me in person at <a href="http://linux.conf.au/">linux.conf.au</a>, in cash or beer form, or I will buy the lucky person a piece of FOSS memorabilia (<a href="https://shop.canonical.com/">example</a>) to that value.</strong></p>
<p>That means fixing the above bug, deleting autofoo files that don't do anything, checking that the translation stuff is setup optimally and generally making it less shameful and easier to follow. The person will also receive additional love points if they implement a skeleton autotools setup for <a href="http://library.gnome.org/devel/gnome-doc-make/stable/migrating.html.en">documentation</a> and for <a href="http://live.gnome.org/ThemableAppSpecificIcons">themed application icons</a>. Remember, I hate autotools this much...</p>Conduit Is Online Desktop2007-08-16T22:10:34+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-08-16:2007/08/16/conduit-is-online-desktop/<p>After a short hacking hiatus, soul searching, and general uncertainty regarding <a href="http://www.conduit-project.org/">Conduits</a> place in the Gnome Online Desktop plan, we the Conduit developers have decided to keep doing what <a href="http://www.conduit-project.org/timeline">we do best</a>; <strong>adding awesome features</strong>, and making Conduit into the best desktop synchronization service for the free desktop. I invite you all to spot the new features in the image below;</p>
<p><a href="http://http://www.johnstowers.co.nz/blog/?attachment_id=110"></a></p>
<p><img alt="Conduits New Features" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-new-features-small.png" /></p>
<p>Going through the new features one by one;</p>
<ul>
<li>
<p>The <em>Network</em> datasink - thats how you tell conduit to share data to other people on the network</p>
</li>
<li>
<p>The <em>chad.local</em> dataprovider - ... and thats how you'll choose to consume other peoples shared data</p>
</li>
<li>
<p>The <em>evo2-sync</em> dataprovider - before long you'll be able to use opensync plugins in conduit</p>
</li>
<li>
<p>The <em>f-spot</em> dataprovider - before long that will be two-way and networkable, work is ongoing with upstream</p>
</li>
<li>
<p><em>Always-Up-To-Date</em> is on the horizon (i.e. auto update and sync changed data)</p>
</li>
<li>
<p><em>iPod photos</em> support - our patches are accepted upstream, so soon we will support those too</p>
</li>
<li>
<p>The <em>facebook</em> dataprovider - just photos so far, but more facebook features will be exposed soon</p>
</li>
</ul>
<p>Other than those <strong>Massive New Features</strong>, there has been continued bug fixing and some improvements to the DBus interface to make it more <a href="http://log.ometer.com/2007-05.html#17">Havoc Compliant</a>. Check out the sample below for how to use Conduits DBus interface to synchronize things...</p>
<div class="codehilite"><pre><span></span>bus = dbus.SessionBus()
#Create an Interface wrapper for the remote object
remote_object = bus.get_object(APPLICATION_DBUS_IFACE,"/")
app = dbus.Interface(remote_object, APPLICATION_DBUS_IFACE)
#get a dataprovider
path = app.GetDataProvider("TestSource")
source = dbus.Interface(
bus.get_object(DATAPROVIDER_DBUS_IFACE,path),
DATAPROVIDER_DBUS_IFACE
)
#get another
path = app.GetDataProvider("TestSink")
sink = dbus.Interface(
bus.get_object(DATAPROVIDER_DBUS_IFACE,path),
DATAPROVIDER_DBUS_IFACE
)
#get a conduit
path = app.BuildConduit(source, sink)
cond = dbus.Interface(
bus.get_object(CONDUIT_DBUS_IFACE,path),
CONDUIT_DBUS_IFACE
)
print "Synchronize the conduit"
cond.Sync()
</pre></div>
<p>I have written an <a href="http://www.conduit-project.org/browser/trunk/tools/example-dbus-conduit-client.py">example</a> which shows how to use Conduit to upload photos to any of our supported photo sites with even fewer lines of code. The DBus API documentation lives at the top of <a href="http://www.conduit-project.org/browser/trunk/conduit/DBus.py">this file</a>. Dont forget that by using a desktop wide sync framework you get all the Conduit bling for free, including</p>
<ul>
<li>
<p>Don't upload duplicates</p>
</li>
<li>
<p>Newer photos replace older ones</p>
</li>
<li>
<p>Asynchronous and all authentication is handled by Conduit</p>
</li>
</ul>Holiday Debrief2007-07-23T07:03:16+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-07-23:2007/07/23/holiday-debrief/<p>Well I have finally arrived in Canada, which signifies the end of my travels around Europe. Back in NZ in a week to start my PhD. Aside from the photos which I have been taking and <a href="http://www.flickr.com/photos/nzjrs/">irregularly uploading</a> I have;</p>
<ul>
<li>
<p>Bought a Nokia n800 (watch for a conduit port soon)</p>
</li>
<li>
<p>Played with an iPhone (its the little things, attention to detail of all transitions between operations)</p>
</li>
<li>
<p>Watched Transformers (not bad)</p>
</li>
<li>
<p>Read (1984, Fahrenheit 911, The God Delusion)</p>
</li>
<li>
<p>Hacked on Conduit</p>
</li>
<li>
<p>Missed GUADEC (I hope the videos and slides will go up soon)</p>
</li>
<li>
<p>Updated to Gutsy (went smoothly)</p>
</li>
</ul>
<p><strong>Conduit Schmonduit</strong></p>
<p>I <a href="http://www.conduit-project.org/wiki/0.3.2">released 0.3.2</a> last week or so. Aside from bug fixes we also added <a href="http://www.box.net/">box.net</a> support (sync evolution/files/tomboy notes/etc to/from/via box.net), and also added the ability to delete photos from the supported photo sites (Flickr, picasaweb, smugmug). This means that Conduit can now better keep your photos (in a folder, or in f-spot) in sync. This includes removing them when they are removed from the synchronized folder, or tagged/untagged in F-spot.</p>
<p>Most of the work is now happening in the <a href="http://www.conduit-project.org/browser/branches/better-mvc">better-mvc</a> branch which will land soon. This is a general tidy up of the code base and a better split of the synchronization logic from the GUI. This will not only allow for improved good looks; but other improvements too.</p>
<p><img alt="Conduit Mockup" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-mockup.png" />
<em>Mock-up Credit: <a href="http://www.qdh.org.uk/">Karl Lattimer</a></em></p>Conduit Network Sync2007-06-12T04:34:42+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-06-12:2007/06/12/conduit-network-sync/<p>Excuse the shameless quoting of fellow conduit contributor <a href="http://unrouted.co.uk/2007/06/10/conduit-network-sync/">John Carr's blog post</a> but this <a href="http://video.google.co.uk/videoplay?docid=-2623798477165899506&pr=goog-sl">video</a> is pretty cool;</p>
<p>[googlevideo]http://video.google.co.uk/videoplay?docid=-2623798477165899506&pr=goog-sl[/googlevideo]</p>
<p><em>Video showing how you can use Conduit to synchronise Evolution Contacts and Tomboy Notes between two machines running the GNOME desktop environment.</em></p>
<p><em>To start with, laptop 1 has 2 contacts and 1 tomboy note. Laptop 2 just has 1 tomboy note. After syncing, both will have 2 contacts and 2 notes. Not too exciting.. but then... (on laptop 2) 1 note and 1 contact are deleted. The remaining note and contact are modified as well. When the camera flips back to laptop 1 and the sync happens you will see the windows update in real time - including the an <em>open</em> tomboy note getting updated.</em></p>
<p><em>Apologies for the poor quality</em></p>Misc Updates2007-06-07T11:34:36+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-06-07:2007/06/07/misc-updates/<p><strong>Conduit</strong></p>
<p>For all those that are interested I just released <a href="http://www.conduit-project.org/wiki/0.3.1">Conduit 0.3.1</a>, the second development release in the series. The release includes a heap of bug fixes and some interesting new features including;</p>
<ul>
<li>
<p>Evolution Support (memos and tasks) using <a href="http://www.conduit-project.org/wiki/evolution-python">evolution-python</a></p>
</li>
<li>
<p>Ipod calendar and tasks support</p>
</li>
<li>
<p>Support <a href="http://www.smugmug.com/">SmugMug</a> and <a href="http://picasaweb.google.com/">Picasaweb</a></p>
</li>
</ul>
<p>Goals for the next release include</p>
<ul>
<li>
<p>Supporting box.net</p>
</li>
<li>
<p>Google notes support</p>
</li>
<li>
<p>100% test coverage of the core sync engine</p>
</li>
</ul>
<p>The thing that keeps me interested in conduit is the speed at which we are progressing. Each new dataprovider added has a benefit proportional to the number of already included dataproviders; for instance adding evolution support opened up the ability to do tomboy notes <--> evolution memos, as well as evolution memos <--> ipod notes, etc.</p>
<p><strong>Gnome Link Drop</strong></p>
<p>Here are some interesting things on the horizon that made me excited about GNOME and gtk. Delegated to people with more free time and ability than me!</p>
<ul>
<li>
<p>Increased activity on <a href="http://bugzilla.gnome.org/show_bug.cgi?id=318807">gtk offscreen rendering</a>. Great for canvas implementors and bling in the future.</p>
</li>
<li>
<p><a href="http://bugzilla.gnome.org/show_bug.cgi?id=119189">OpenGl support in Gtk</a></p>
</li>
<li>
<p>Saw <a href="http://zrusin.blogspot.com/2007/06/mirroring-widgets.html">this</a> <a href="http://labs.trolltech.com/page/Graphics/Examples/Examples2">example</a> from the folks over at KDE. Made with the help of <a href="http://ktown.kde.org/~fredrik/composite_howto.html">this documentation</a>. Perhaps someone with more ability than me can use this as the basis for <a href="http://bugzilla.gnome.org/show_bug.cgi?id=78040">window previews in libwnck</a>.</p>
</li>
<li>
<p><a href="http://bugzilla.gnome.org/show_bug.cgi?id=444659">GtkTimeline</a> <a href="http://blogs.gnome.org/view/carlosg/2007/06/06/0">bling</a>, lets make <a href="http://www.jonobacon.org/?p=930">organic interfaces</a> easy.</p>
</li>
<li>
<p>AFAICT <a href="http://bugzilla.gnome.org/show_bug.cgi?id=412882">Composited gtk/gdk widgets</a> makes <a href="http://blogs.gnome.org/view/ryanl/2007/02/18/0">new panel</a> possible?</p>
</li>
</ul>
<p>Anyway, Im still on holiday, and loving it!</p>
<p><img alt="" src="http://farm1.static.flickr.com/199/499905707_1d11d5a77b_m.jpg" /></p>Evolution and Conduit Action2007-06-01T08:03:47+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-06-01:2007/06/01/evolution-and-conduit-action/<p>The reaction to my <a href="http://www.johnstowers.co.nz/blog/index.php/2007/05/30/python-bindings-for-evolution/">previous post</a> was extremely positive, with a lot of interest in the bindings. Ive had a bit more free time and done some more hacking on both the bindings, and integrating evolution support into <a href="http://www.conduit-project.org/">Conduit</a>. On the first point;</p>
<ul>
<li>
<p><a href="http://www.conduit-project.org/wiki/evolution-python">evolution-python-0.0.1</a> is available (<a href="http://files.conduit-project.org/releases/evolution-python-0.0.1.tar.gz">download</a>) (<a href="http://doc.conduit-project.org/evolution-python/">API docs</a>) (<a href="https://svn.greenbirdsystems.com/evolution-python/trunk/test/evo-test.py">example</a>)</p>
<ul>
<li>
<p>Implements add(), remove(), delete() and update() for ECalComponents into ECal sources</p>
</li>
<li>
<p>Implements convenience setters/getters for ECalComponent summary and descriptions (so you don't need to see the underlying ical types)</p>
</li>
<li>
<p>In short you should now be able to build note taking, todo applications, and addressbook managers on top of evolution data-server using python.</p>
</li>
</ul>
</li>
</ul>
<p>Work on the <a href="http://www.conduit-project.org/wiki/0.3.1">0.3.1</a> bugfix release of Conduit is coming along nicely, it should be out sometime over the weekend. In between fixing bugs however I been able to introduce some new dataproviders, which in turn will open up a heap of new synchronization possibilities.</p>
<ul>
<li>
<p>Thanks to new conduit contributor Thomas Van Machelen we now support Smugmug and Picasaweb photo sharing sites.</p>
</li>
<li>
<p>Support for Facebook photos (uploading yours, and downloading your friends) is nearing completion thats to the excellent API complete <a href="http://code.google.com/p/pyfacebook/">pyfacebook bindings</a>.</p>
</li>
<li>
<p>By supporting Evolution Memos and TODO items we now support some pretty cool sync combinations;</p>
<ul>
<li>
<p>Evolution Memos <--> Backpackit.com and Tomboy Notes</p>
</li>
<li>
<p>Evolution Memos and Contacts <--> iPod</p>
</li>
<li>
<p>gnome-about-me (evolution) <--> Facebook profile</p>
</li>
</ul>
</li>
</ul>
<p><a href="http://www.flickr.com/photos/nzjrs/523585817/"><img alt="" src="http://farm1.static.flickr.com/254/523585817_9ba48925c6_m.jpg" /></a> <a href="http://www.flickr.com/photos/nzjrs/523592511/"><img alt="" src="http://farm1.static.flickr.com/243/523592511_7e16dea003_m.jpg" /></a></p>
<p>I recognise not all of these sync pathways are immediately useful or new, but they are providing a good test of the framework, checking if the design is correct and providing useful test cases. Its also exciting to think that when <a href="http://www.conduit-project.org/wiki/SpecNetworkSupport">network support</a> lands next cycle, direct sync between two computers over the web/LAN will just work, and will support any dataprovider that is available to conduit.</p>Python Bindings for Evolution2007-05-30T05:34:33+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-05-30:2007/05/30/python-bindings-for-evolution/<p>In order to add <a href="http://www.go-evolution.org/">evolution</a> support (contacts, task items and memos) to <a href="http://www.conduit-project.org">Conduit</a> I needed to be able to access the relevant parts of evolution-data-server from python. <a href="http://www.conduit-project.org/wiki/evolution-python">evolution-python</a> is an incomplete wrapper around <a href="http://www.gnome.org/projects/evolution/developer-doc/libebook/">libebook</a> and <a href="http://www.gnome.org/projects/evolution/developer-doc/libecal/">libecal</a>. It has now got to the stage of being useful, so here is a somewhat nervous announcement of evolution-python v0.0.0</p>
<ul>
<li>
<p><a href="http://files.conduit-project.org/releases/evolution-python-0.0.0.tar.gz">Download</a></p>
</li>
<li>
<p><a href="http://doc.conduit-project.org/evolution-python/">API Documentation</a></p>
</li>
</ul>
<p>This should be considered a work in progress and will be changing a lot over the next few days as I implement methods needed for the two way synchronization of evolution data using Conduit. However the basic skeleton is there, and I invite people to help with the bindings. An easy way to access evolution-data-server from python has been missing for a while now so I hope evolution-python will be useful for others.</p>
<p>There are some known issues and items to work on listed on the <a href="http://www.conduit-project.org/wiki/evolution-python">project website</a>. Here is a small screenshot of the included test application.</p>
<p><img alt="" src="http://farm1.static.flickr.com/200/520079516_ebe20666f6_m.jpg" /></p>Finally A Release2007-05-07T04:24:26+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-05-07:2007/05/07/finally-a-release/<p>Finally, and <em>?only?</em> 3 months overdue I have released <a href="http://www.conduit-project.org/wiki/0.3.0">Conduit 0.3.0</a>. This release finally marks the end of the sync engine rewrite from the previous release. This also signifies the first time that conduit is simultaneously useful to end users, and application developers as a desktop sync service.</p>
<p>[<img alt="Screenshot-Conduit-0-3-0-1.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/Screenshot-Conduit-0-3-0-1.png" /> <img alt="Screenshot-Conduit-0-3-0-2.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/Screenshot-Conduit-0-3-0-2.png" />](http://www.conduit-project.org/wiki/Screenshots)</p>
<p><strong>What can it do?</strong></p>
<p>From an end user perspective Conduit has reached the level of being useful. I am currently travelling around Europe for a few months and using Conduit on a daily basis, at least for the task of Synchronization/backup of my photos to Flickr and my home server, Sync/Export of my Tomboy notes to iPod. Other than that Conduit can currently perform the following sync partnerships;</p>
<ul>
<li>
<p>Two way file/folder sync on gnomevfs volumes</p>
</li>
<li>
<p>Two way Tomboy note sync via gnomevfs volumes</p>
</li>
<li>
<p>Two way Tomboy note sync via ipod notes</p>
</li>
<li>
<p>One way sync of files/folders of photos to Flickr</p>
</li>
<li>
<p>One way sync of FSpot tagged photos to Flickr</p>
</li>
</ul>
<p>In the one way sync/export case Conduit is smart in the sense that if a piece of data has not been modified then it will not be synchronized/exported again, it will replace the existing data.</p>
<p>I have also added conflict detection, and a UI for resolving conflicts, including the ability to compare the conflicting data (using gnome-open on the relevant URI).</p>
<p><strong>Desktop Sync Service</strong></p>
<p>I have been talking about the merits of Sync as a desktop sync service for a while, and this release finally marks the point where I can start to add export and sync capabilities to GNOME apps using conduit. In this situation Conduit is exclusively controlled via DBus (independent of the UI). By sharing the same DB we can avoid duplicate data in the synchronization process.</p>
<p>To demonstrate this I modified to <a href="http://micampe.it/projects/flickruploadr">Uploadr</a> to call conduit over DBus (and called the resulting app <a href="http://www.conduit-project.org/browser/trunk/tools/yaput/Yaput">Yaput - yet another photo upload tool</a>) (<a href="http://www.conduit-project.org/attachment/wiki/Screenshots/Screenshot-Uploadr.png">screenshot</a>). This means that duplicate photos will not be uploaded, updated photos will replace old ones, and it doesn't matter whether you upload the photos from the Conduit GUI, the DBus interface or $(YOUR_APP_HERE). All that using 10 DBus calls!</p>
<p>Anyway, that's the plan, and apart from a few annoying <a href="http://www.conduit-project.org/wiki/0.3.0">known issues</a>, I finally feel like I am on track.</p>
<ul>
<li>
<p><a href="http://www.conduit-project.org/wiki/Releases">Download</a></p>
</li>
<li>
<p><a href="http://www.conduit-project.org/newticket">Report Bugs</a></p>
</li>
<li>
<p>Good Luck!</p>
</li>
</ul>Life Updates2007-05-04T07:59:40+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-05-04:2007/05/04/life-updates/<p>So I thought I should spice up my blog with a post about non nerdy stuff. Currently I am travelling around Europe and will continue doing that until about June/July at which time I fly to Canada.</p>
<p><img alt="Prague" src="http://www.johnstowers.co.nz/blog/static/images/imported/PraguePanorama.jpg" /></p>
<p>I was initially planning on doing a University exchange at the University of British Columbia but have since decided to take up a PhD at the <a href="http://www.canterbury.ac.nz/">University of Canterbury</a>. I will (hopefully) be working on <a href="http://www.albatross-uav.org/index.php/Main_Page">UAV stuff</a>, likely machine vision related, in the <a href="http://www.grcnz.com/">Geospatial Research Centre</a> at the university, beginning on the 1st of August.
Travelling by car across Europe by car is so far been exceptionally awesome. Have me some cool characters and have certainly become a better driver. It has allowed us the freedom to go where we want, and when split three ways, has been just as economic as train travel.</p>
<p><img alt="Croatia" src="http://www.johnstowers.co.nz/blog/static/images/imported/Croatia.jpg" /></p>Conduit Updates2007-03-29T05:45:53+02:00John Stowerstag:www.johnstowers.co.nz/blog,2007-03-29:2007/03/29/conduit-updates/<p>Its been a while since I blogged about <a href="http://www.conduit-project.org">Conduit</a>, and a lot has been going on. My main task has been working on the conflict resolution UI and on tweaking the conversions between datatypes so as to maintain maximum fidelity. <a href="http://www.unrouted.co.uk/">John Carr</a>, the other main Conduit contributor has also been working on some awesome stuff.</p>
<p><strong>Conflict Resolution</strong></p>
<p>During the synchronization process a conflict can occur in any of the following scenarios</p>
<ol>
<li>
<p>In a one way synchronization, if the destination data is newer that the source data (the <em>exported data shoud not clobber user updated data</em> case)</p>
</li>
<li>
<p>In a two way sync when the destination data has been modified since the last synchronization (the <em>two computers sync to the same location but both have had source changes</em> case)</p>
</li>
<li>
<p>In any situation when (for backend specific reasons) a comparison between data is unable to determine which is newer than the other.</p>
</li>
<li>
<p>A piece of source data has been deleted and the user has selected a synchronization policy in effect which states they want to be certain before deleting the last remaining copy of that data at the destination. (I handle delete as a special case of a conflict)</p>
</li>
</ol>
<p>Dataprovider backends communicate conflicts to the UI via signals. The UI has access to the conflicting data so can offer the user the ability to compare which is newer. Conflicts use the same arrow metaphor that the main UI uses and can either be resolved in the main window, or be shown with more detail in their own window.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/ConduitConflict.png"><img alt="ConduitConflict.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/ConduitConflict.thumbnail.png" /></a><a href="http://www.johnstowers.co.nz/blog/static/images/imported/ConduitConflictWindow.png"><img alt="ConduitConflictWindow.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/ConduitConflictWindow.thumbnail.png" /></a></p>
<p>If the data has set a (gnome-open'able) URI then the user can view the data at this location and decide which is newer. For example, a user is synchronizing his Tomboy notes via an iPod. If there is a conflict in this scenario then clicking Compare will cause Tomboy to show the local note and will launch a text editor to inspect the conflicting note off the iPod. <em>(Lazyweb request: Any idea on how I might arrange the windows side-by-side via libwnck for example).</em></p>
<p><strong>Fidelity</strong></p>
<p>The main difficulty with synchronizing different datatypes, or the same datatypes but via a middleman is maintaining fidelity (that is not losing any information) during the whole process. In the case of contact or calendar data then the default would be to support just standard datatypes (vcard, ical, etc).</p>
<ul>
<li>
<p><a href="http://www.opensync.org/wiki/devel-branch-0.30">Opensync 0.30</a> is looking to define their own XML schema for basic datatypes, and I would support this idea and likely adopt the same schema.</p>
</li>
<li>
<p>Apple .mac uses strict datatypes and pushes everything to a central server. One alternative to maintaining fidelity is to keep data in all formats but maintain a single canonical source of modification times, and perform conversions (even if they lose fidelity) only when needed from this central store.</p>
</li>
</ul>
<p>This leads onto my next suggestion Having an optional central repository (like .mac) not only addresses some of the difficulties of maintaining fidelity through conversion but also opens up some other use-cases which the open source world has not addressed.</p>
<ul>
<li>
<p>Online editing of tomboy notes while also being able to sync them on multiple computers (I can currently do this through backpackit.com but the conversions lose fidelity/information)</p>
</li>
<li>
<p>Synchronize desktop settings (wallpaper, theme, nautilus prefs, etc between two computers)</p>
</li>
<li>
<p>One click file sync with no configuration.</p>
</li>
</ul>
<p>So lazywab hackers. Who here is a <a href="http://www.djangoproject.com/">Django</a> hacker that wants to write me a web service to do this. I already have a documented API that you could write against. I have the spec, the idea, but not the time. I had considered writing this myself, and setting it up as a paid service (I would pay a few dollars a month to be able to do this) but dont have the time, nor the web development experience, nor an inclination of artistic ability to make it look nice! Any takers, please email me!</p>
<p><strong>Miscellany
</strong></p>
<ul>
<li>
<p>John has finished up all the remaining pieces to allow iPods to be <a href="http://unrouted.co.uk/2007/03/19/hot-removals/">hotplugged/removed</a> in conduit, so that synchronization settings and state are preserved.</p>
</li>
<li>
<p>He is also hacking on direct computer-computer sync over local network. Using avahi for discovery and pickling data over the wire. Should be awesome when this lands!</p>
</li>
<li>
<p>Unfortunately I was not eligible for summer of code this year, I am on <a href="http://www.flickr.com/photos/nzjrs/">holiday</a> from university for six months and wont be starting PhD till after the summer. I was pleased to see mention of Conduit in the <a href="https://wiki.ubuntu.com/GoogleSoC2007">Ubuntu SOC ideas page</a>, so I am glad to see distros recognizing the importance of making synchronization easy!</p>
</li>
<li>
<p>Releasing 0.3.0 is blocking on my having a working Ubuntu Fesity install to test against. <em>(Lazyweb request: Does suspend,resume and hotkeys for panasonic sub notebooks such as the CF-R4 work on Feisty out of the box - it has not done so in previous releases)</em></p>
</li>
</ul>
<p><em>Update: <a href="http://googledataapis.blogspot.com/2007/03/snakes-on-gdata-announcing-python.html">Google just released python GData bindings</a>. This will make excellent support for Google Calendar, Google Notebook and a few other ideas I have up my sleeve a reality in the near future!. My blog is also back up, sorry for the trouble.
</em></p>Threading and (py)Gtk2007-03-12T02:58:31+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-03-12:2007/03/12/threading-and-pygtk/<p>At the request of a few people I have made a <a href="http://gist.github.com/51686">demo to share</a>, showing how I do threading in a pygtk application (<a href="http://www.conduit-project.org">Conduit</a>). I find the following approach seems to work reliably and requires little code. Other approaches can be found <a href="http://aruiz.typepad.com/siliconisland/2006/04/threads_on_pygt.html">here</a> and <a href="http://live.gnome.org/PyGTK/FAQ/MainLoopAndThreading">here</a>.</p>
<p>This approach takes advantage of the fact that signal emission in glib has been threadsafe since glib 2.8 (IIRC). All communication with the GUI is done via gobject signals. There is definitely a compromise in the level of GUI fiddling that you can do, particularly when compared with the <em>threads_enter/leave</em> approach. However, I have found this signal based approach sufficent in my case, and that it encourages me to decouple the slow blocking tasks from the GUI.</p>
<p>A lot of the tasks in conduit take a long time (network limited), and there is no real need for extensive GUI interaction with them once they have been started. I am really only interested in their progress, and when they complete. With this in mind I have implemented the following approach;</p>
<ul>
<li>
<p><em>FooThreadManager</em>
This class is the entry point for starting threads (<em>the make_thread() method</em>) . Its basically just a threadpool that starts threads with the appropriate arguments, while restricting the number of concurrent running threads below a user defined limit. It also connects the threads to the supplied user callbacks.</p>
</li>
<li>
<p>_IdleObject
Like a normal gobject.GObject but emits all signals in the main thread</p>
</li>
<li>
<p>__FooThread_
A simple class which derives from both threading.Thread and _IdleObject. All work is done in run() and a signals are emitted when the thread completes and to show progress</p>
</li>
<li>
<p><em>Demo</em>
A simple demo (see screenshot) which can start a whole bunch of threads and receive notification when they complete.</p>
</li>
</ul>
<p><img alt="FooThread Test Program" src="http://www.johnstowers.co.nz/blog/static/images/imported/screenshot-foothread.png" /></p>
<p>Anyway, the <a href="http://gist.github.com/51686">Example Code</a> is a bit contrived and will certainly need some customization by the user but nonetheless may still be useful to others.</p>
<p><strong>Update:</strong> Thanks to comments I fixed up a thread-safety issue. I had misunderstood that signal handlers get run immediately in emit(). Now the code emit()s on an idle_handler so all signals and callbacks are run in the main thread. As i mentioned, I use this approach in situations where the threads may run and block for a long time, so the burden of processing all the signals in the main thread is not a big deal as they do not occur frequently.</p>
<p><strong>Update 2:</strong> Added progress reporting</p>Conduit Architecture and Updates2007-03-10T02:31:41+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-03-10:2007/03/10/conduit-architecture-and-updates/<p>A lot has been happening on the <a href="http://www.conduit-project.org/">Conduit</a> front of late, and I probably should have blogged about it earlier. Unfortunately, it has seemed as though I have made 5 steps forward and 4 steps back with every hour of work on the project, so why, and whats up?</p>
<h3>Background</h3>
<p>I will be dropping a 0.3 release to coincide with the release of GNOME 2.18. This will be a developer release designed to get things working well before a stable 0.4 release some time after.</p>
<p>All of Conduits synchronization capability is determined at runtime. Conduit scans the users' and the system wide dataproviders directories in the same way that <a href="http://raphael.slinckx.net/deskbar/">Deskbar</a> loads handlers when it starts. The core synchronization logic is decoupled from the dataproviders, and only those relevant dataproviders are shown to the user (i.e. only show iPod Notes if an iPod is connected, etc).</p>
<p>Remember that I want Conduit to be a GNOME wide synchronization service that other applications can hook into, so I need to not only decouple the core sync logic from the application and usage specific dataprovider back ends, but also allow conduit to be extended by third parties.</p>
<h3>Extension by 3rd Party Applications</h3>
<p>I mentioned that all synchronization capability is determined at runtime. The other half of this story is that a third party may wish to synchronize something that Conduit knows nothing about (I'll use a <a href="http://www.jokosher.org/">Jokosher</a> project as an example, although in my next post I will address these issues in the Tomboy case). Lets assume that a Jokosher project cannot be described in terms of the basic types of data that Conduit knows about (email, contact, file, etc). In order to allow Jokosher projects to be synchronized between different computers the Jokosher developers would need to do the following</p>
<ol>
<li>
<p>Define a Jokosher <a href="http://www.conduit-project.org/browser/trunk/conduit/datatypes/DataType.py">datatype</a> that describes what it is to be synchronized.</p>
</li>
<li>
<p>(optionally) Define some conversions between the Jokosher datatype and the basic Conduit datatypes (File, Text, etc)</p>
</li>
<li>
<p>Define a <a href="http://www.conduit-project.org/browser/trunk/conduit/DataProvider.py">dataprovider</a> that encapsulates Jokosher invocation specific information such as the Jokosher user, and how does one go about extracting/modifying Jokosher specific data when Jokosher may be running.</p>
</li>
</ol>
<p>The Conduit framework will do a lot of heavy lifting here, such as caching modification times, providing a way for the user to configure a Jokosher sync partnership, providing a way for a user to resolve conflicts, and even allowing the whole sync process to be controlled from outside of the application over DBus. Most interestingly however, depending on the fidelity and number of conversions specified in #2, Conduit will facilitate a number of other useful sync scenarios including,</p>
<ul>
<li>
<p>Sync Jokosher projects via any gnomevfs compatible location (ftp, etc)</p>
</li>
<li>
<p>Sync via <a href="http://www.amazon.com/gp/browse.html?node=16427261">Amazon S3</a> (coming soon)</p>
</li>
<li>
<p>Sync via USB key</p>
</li>
<li>
<p>Sync directly over a local network using Avahi. (next release)</p>
</li>
</ul>
<p>The point here is that this decoupling of the dataproviders, from the core and the datatypes (sans the conversion functions) lets the Jokosher dataprovider take advantage of any/all additional <em>$FUTURE</em> dataproviders (e.g. Amazon S3) and all improvements to the Conduit core (such as direct sync using pickle'd python over network via Avahi).</p>
<h3>Testing</h3>
<p>As I get closer to release I get more nervous about releasing a tool which <strike>will</strike> could eat peoples data. To mitigate this risk I spent a week or two implementing an extensive testing framework for Conduit, a task I should have tackled much earlier in the project.</p>
<p>Unfortunately the testing is made more complicated when the applications capabilities can change between invocations (dynamic loading of dataproviders and conversions explained earlier). Aside from the usual unit level tests (does this throw the correct exceptions, etc) I am also required to test the fidelity and accuracy of the conversions between datatypes (round trip comparisons), and the fidelity of synchronization from dataprovider <em>$FOO</em> to dataprovider <em>$BAR</em> via any of these datatypes.</p>
<p>The testing framework uses a mixture of <a href="http://www.conduit-project.org/browser/trunk/test/python-tests">python</a> and <a href="http://www.conduit-project.org/browser/trunk/scripts/run-tests.sh">shell script</a>, and is even web 2.0 compliant ;-). It currently tests available conversions, saving and getting data from various dataprovider back ends (Flickr, gnomevfs, Tomboy, etc). It even does <a href="http://tests.conduit-project.org/coverage/">code coverage</a> and pushes the <a href="http://tests.conduit-project.org/">results on line</a>.</p>
<h3>What's Next</h3>
<p>I apologize for this post being mostly academic in nature, lacking in screen shots. I will post again in a few days with some more relevant information (from a users perspective). I just wanted to highlight how and why I believe a desktop sync service is useful, and how I am trying to satisfy this in a sustainable way for the GNOME platform as a whole.</p>
<p>I anticipate finishing up the two way Tomboy support (via iPod notes, gnomevfs location, and <a href="http://backpackit.com">backpackit.com</a>) in the next few days. There are some bugs preventing running Conduit on Feisty (<a href="http://mail.python.org/pipermail/python-list/2007-February/427715.html">goocanvas</a> 0.6.0 is ABI incompatible with 0.4.0). There are also some other lose ends (like some GUI blocking calls in conflict resolution, and some difficulties saving program configuration). Unfortunately, like a lot of FOSS, the best is always just around the corner....</p>Another Introductory Post2007-02-19T19:47:28+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-02-19:2007/02/19/another-introductory-post/<p>Hey Planet GNOME'rs</p>
<p>Thanks <a href="http://perkypants.org/blog/">Jeff</a> for putting me on Planet GNOME. My name is John Stowers and i'm currently splitting my time between a few things that might be of interest to people here.</p>
<ol>
<li>
<p><strong>Synchronization and the GNOME Desktop</strong>
I probably spend most of my free time hacking on Conduit, which is a synchronization application for GNOME. I hope to provide a (DBus) service where application authors can use Conduit for their individual sync and export capabilities, and don't have to keep reimplementing them in their own applications. Furthermore as a stand alone application I aspire to the ease of use of Apple's .Mac while supporting core GNOME technologies.
<img alt="Conduit pre 0.3.0" src="http://www.johnstowers.co.nz/blog/static/images/imported/Conduit-Near-v0-3-0.png" />
More information</p>
<ol>
<li>
<p><a href="http://www.conduit-project.org/">Conduit Website
</a></p>
</li>
<li>
<p><a href="http://www.johnstowers.co.nz/files/presentations/LCA2007_GNOME_Conduit_Presentation.pdf">Slides</a> and <a href="http://mirror.linux.org.au/pub/linux.conf.au/2007/video/monday/monday_1450_GNOME.ogg">Video</a> from a <a href="http://live.gnome.org/Sydney2007#head-9b2b1f3065cf8635338348f7f2d4076dc744c054">talk</a> at <a href="http://lca2007.linux.org.au/">Linux.conf.au</a></p>
</li>
<li>
<p>The next release is scheduled for around the same time as the release of GNOME 2.18 and will support that favorite requested feature <em>cough</em>tomboy synchronization<em>cough</em>. <a href="http://blog.timothy.ws/2007/02/whats-new-for-tomboy-060-not-released.html">Thanks Boyd</a>.</p>
</li>
</ol>
</li>
<li>
<p><strong>A Metadata Enabled GNOME</strong>
Im pretty excited about <a href="http://www.gnome.org/projects/tracker/">Tracker</a> and the ability to move away from a folder centric GNOME. By embracing metadata, tagging, and a desktop indexer I think that we will leap past our competitors in terms of integration between applications. To this end I am hacking on;</p>
<ol>
<li>
<p>Nautilus + Tracker integration
Injecting some love into the nautilus emblem functionality. Support tagging and attaching emblems (emblem = a tag with an image) from either nautilus or Tracker.</p>
</li>
<li>
<p>Nautilus metadata integration
Working with <a href="http://njpatel.blogspot.com/">neilj</a> on a nautilus-on-steroids + libtracker-gtk. Lets make metadata visible (in nautilus) and easy for application authors to add to their apps (libtracker-gtk)</p>
</li>
<li>
<p>GtkFileChooser + Tracker
Attach tags and notes at save time (whats the state of the <a href="http://primates.ximian.com/~federico/docs/file-chooser-extension-spec/">filechooser extension spec</a>?)</p>
</li>
</ol>
</li>
</ol>
<p>Other than that Im 23 years old, just finished a <a href="http://www.mem.canterbury.ac.nz/">masters</a> and <a href="http://www.elec.canterbury.ac.nz/">electrical engineering at the University of Canterbury</a>. Im really an electronics nerd so im also hacking on <a href="http://www.albatross-uav.org/index.php/Main_Page">Albatross Unpiloted Aerial Vehicle</a>. Im doing some traveling around Europe for the next few months and then probably starting a phD.</p>A Metadata Enabled GNOME2007-02-06T13:41:27+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-02-06:2007/02/06/a-metadata-enabled-gnome/<p>When <a href="http://www.gnome.org/projects/tracker/">Tracker</a> was proposed for GNOME 2.18 I was one of its <a href="http://mail.gnome.org/archives/desktop-devel-list/2007-January/msg00290.html">staunchest supporters</a>, arguing that a GNOME wide unified metadata storage system would enable a richer desktop experience, and take GNOME beyond its competition. Tracker did not make the cut for GNOME 2.18, and will no doubt be proposed again for GNOME 2.20.</p>
<p>To help people see the potential of a GNOME desktop using Tracker I have been working on two projects for the past little while. Both of these are components of my <a href="http://www.johnstowers.co.nz/wiki/index.php/Tracker_Bling">larger vision</a> of a metadata rich GNOME desktop. These initial attempts just focus on making tagging [1] a more consistent experience for all GNOME apps. Consider the examples and screenshots (for a file called nice) below;</p>
<ul>
<li>
<p><a href="http://www.freedesktop.org/wiki/Standards_2fdesktop_2demblem_2dspec">Freedesktop emblem spec</a>
<em> Allows desktop file managers and indexers to present a list of predefined emblems and tags to the user, and allows these predefined emblems/tags to be installed by third party developers in a consistent manner.</em></p>
</li>
<li>
<p>Tracker Nautilus Integration
<em>Nautilus using tracker for storage of all tags and emblems.</em></p>
</li>
<li>
<p>libtracker-gtk
<em>A bunch of gtk widgets that application authors can use to add tracker functionalit to their application.</em></p>
</li>
</ul>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/libtracker-gtk-nautilus.png"><img alt="Nautilus Tracker Intergration" src="http://www.johnstowers.co.nz/blog/static/images/imported/libtracker-gtk-nautilus.thumbnail.png" /> </a><a href="http://www.johnstowers.co.nz/blog/static/images/imported/libtracker-gtk-2.png"><img alt="A Tracker connected GtkListStore" src="http://www.johnstowers.co.nz/blog/static/images/imported/libtracker-gtk-2.thumbnail.png" /></a> <a href="http://www.johnstowers.co.nz/blog/static/images/imported/libtracker-gtk-1.png"><img alt="A GtkEntryCompletion widget" src="http://www.johnstowers.co.nz/blog/static/images/imported/libtracker-gtk-1.thumbnail.png" /></a></p>
<p>Industrious individuals could probably find the bzr repositories for the above work, but at the moment it is not quite ready for prime time consumption. Stay tuned for more news.</p>
<p>[1]<em> Terminology:</em></p>
<ul>
<li>
<p><em>Tag: A descriptive keyword the user has associated with a file</em></p>
</li>
<li>
<p><em>Emblem: A tag with an associated image</em></p>
</li>
</ul>Bling to Get Excited About2007-01-26T01:09:57+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-01-26:2007/01/26/bling-to-get-excited-about/<p>Some interesting things have popped up in the FOSS world in the last few weeks. Some of you may have seen these, others may not;</p>
<ul>
<li>
<p>XFCE got some substancial <em>bling</em></p>
<ul>
<li>
<p><a href="http://foo-projects.org/~benny/articles/xfce44-visual-tour.html">XFCE 4.4 screenshot tour</a> - look at the compositing window manager<img alt="
" src="http://foo-projects.org/~benny/articles/xfce44-visual-tour.html" /></p>
</li>
<li>
<p><a href="http://linux.wordpress.com/2007/01/25/excellent-overview-of-xfce-44/">Another tour</a></p>
</li>
<li>
<p><a href="http://goodies.xfce.org/projects/applications/start">A nice list of XFCE goodies</a></p>
</li>
</ul>
</li>
<li>
<p>New GNOME <em>bling</em></p>
<ul>
<li>
<p><a href="http://blogs.gnome.org/view/lucasr/2007/1/22/0">Eye of GNOME improvements</a></p>
</li>
<li>
<p><a href="http://lunapark6.com/?p=2728">New Control Center shell</a></p>
</li>
<li>
<p><a href="http://linuxfud.wordpress.com/2006/11/25/gnome-theme-manager-to-support-color-scheme-customization/">Color scheme support</a></p>
</li>
<li>
<p><a href="http://svn.gnome.org/viewcvs/metacity/trunk/doc/compositor-control.txt?rev=3047">Metacity with composite support</a></p>
</li>
</ul>
</li>
<li>
<p>Distro <em>bling</em></p>
<ul>
<li>
<p><a href="http://twinpeaks.dyndns.org/blog/general/2007/01/24/metisse-is-not-a-3d-desktop">Metisse</a> - I must have this_
_</p>
</li>
<li>
<p><a href="http://www.pulseaudio.org/">PulseAudio</a> <a href="https://www.redhat.com/archives/fedora-desktop-list/2007-January/msg00091.html">in Fedora</a> (<a href="http://lca2007.linux.org.au/talk/211">PulseAudio is cool</a>)</p>
</li>
</ul>
</li>
<li>
<p>Toolkit <em>bling</em></p>
<ul>
<li>
<p><a href="http://codecave.org/?weblog_id=gegl-0.0.4">GEGL</a> (with new <a href="http://codecave.org/?weblog_id=rgegl">ruby demo</a>)</p>
</li>
<li>
<p><a href="http://www.clutter-project.org/blog/?p=6">Clutter v0.2</a></p>
</li>
</ul>
</li>
<li>
<p>Development <em>bling</em></p>
<ul>
<li>
<p><a href="http://www.gtkpod.org/libgpod.html">libgpod</a> with <a href="http://sourceforge.net/project/shownotes.php?release_id=478662&group_id=67873">iPod photo support</a> (great for Conduit)</p>
</li>
<li>
<p><a href="http://lca2007.linux.org.au/talk/301">Telepathy</a> and<a href="http://telepathy.freedesktop.org/wiki/AppConnect"> Application Data Channels</a> (ADCs)</p>
</li>
<li>
<p><a href="http://wiki.mozilla.org/Gtkmozembed">Gtkmozembed (Firefox 3 / Gecko 1.9 based)</a>. Way more power than its predecessors. More on my plans with this later</p>
</li>
<li>
<p><a href="http://gabriel.sourceforge.net/">Gabriel</a> - DBus over SSH (more on this and its uses in Conduit later)</p>
</li>
</ul>
</li>
</ul>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/metacity-bling.png"><img alt="Metacity with composite support" src="http://www.johnstowers.co.nz/blog/static/images/imported/metacity-bling.thumbnail.png" /></a><a href="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-bling.png"><img alt="Conduit status icon and detachable conflict window" src="http://www.johnstowers.co.nz/blog/static/images/imported/conduit-bling.thumbnail.png" /></a></p>Linux.conf.au2007-01-25T02:05:17+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-01-25:2007/01/25/linuxconfau/<p>Well I had a fantastically great time at <a href="http://lca2007.linux.org.au/">linux.conf.au</a>. I was fortunate enough to present two talks at the conference;</p>
<ul>
<li>
<p><a href="http://live.gnome.org/Sydney2007#head-9b2b1f3065cf8635338348f7f2d4076dc744c054">Synchronization and GNOME (at the GNOME Miniconf)</a></p>
<ul>
<li>
<p><a href="http://mirror.linux.org.au/pub/linux.conf.au/2007/video/monday/monday_1450_GNOME.ogg">Video</a></p>
</li>
<li>
<p><a href="http://www.johnstowers.co.nz/files/presentations/LCA2007_GNOME_Conduit_Presentation.pdf">Slides</a></p>
</li>
</ul>
</li>
<li>
<p><a href="http://lca2007.linux.org.au/talk/229">Albatross - The Open UAV</a> (presented at the main conference in conjunction with Hugo)</p>
<ul>
<li>
<p><a href="http://mirror.linux.org.au/pub/linux.conf.au/2007/video/talks/229.ogg">Video
</a></p>
</li>
<li>
<p><a href="http://www.johnstowers.co.nz/files/presentations/LCA2007_Albatross_UAV.pdf">Slides</a></p>
</li>
</ul>
</li>
</ul>
<p>But I think the coolest thing was meeting heaps of interesting (and impressive) people including <a href="http://www.jonobacon.org/">Jono</a>, <a href="http://mugshot.org/person?who=9pfDXd0WvvZwJS">Eugene</a>, <a href="http://www.flock.com/about/410">Manish</a>, <a href="http://andy.brisgeek.com/">Andy</a>, <a href="http://davyd.livejournal.com/205982.html">Davyd</a>, <a href="http://karla.brisgeek.com/">Karla</a>, <a href="http://blogs.gnome.org/nigeltao">Nigel</a>, <a href="http://lca2007.linux.org.au/profile/26">Rafael</a>, etc, etc. I will put some more photos and some of my new <a href="http://en.wikipedia.org/wiki/BHAG_(Goal)">BHAGS</a> for <a href="http://www.conduit-project.org">Conduit</a> and <a href="http://www.albatross-uav.org">Albatross</a> up soon.</p>
<p><img alt="Showing off Albatross at the Openday" src="http://www.johnstowers.co.nz/blog/static/images/imported/albatross-openday.png" /></p>What have I been doing2007-01-06T02:52:25+01:00John Stowerstag:www.johnstowers.co.nz/blog,2007-01-06:2007/01/06/what-have-i-been-doing/<p>So its been a while since I have posted but there is some interesting news to report;</p>
<ul>
<li>
<p><a href="http://www.conduit-project.org/">Conduit</a> has a <a href="http://www.unrouted.co.uk/">new developer</a> - confusingly also named John</p>
</li>
<li>
<p>I am speaking at <a href="http://lca2007.linux.org.au/">Linux.conf.au</a> both at the <a href="http://live.gnome.org/Sydney2007#head-4d130aea68d321d398e97fe6c194a29a6fe496a4">GNOME Miniconf</a> and the <a href="http://lca2007.linux.org.au/talk/229">Main conference</a></p>
</li>
<li>
<p>I have some interesting Topaz style demos to show soon</p>
</li>
<li>
<p><a href="http://www.conduit-project.org/wiki/TODO">Conduit 0.3</a> is going to rock</p>
<ul>
<li>
<p>Opensync will be supported (to some degree)</p>
</li>
<li>
<p>The UI is starting to really ROCK, and improved treeview to display dataproviders, and improved canvas performance.</p>
</li>
<li>
<p>I have been working on a <a href="http://www.conduit-project.org/wiki/evolution-python">python binding to evolution</a>. It is starting to reach the state of being useful and will be supported in the next release. I hope that this library could one day be included in GNOME.</p>
</li>
</ul>
</li>
</ul>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/Conduit-Near-v0-3-0-preview.png"></a></p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/Conduit-Near-v0-3-0.png"><img alt="Conduit-Near-v0-3-0-preview.png" src="http://www.johnstowers.co.nz/blog/static/images/imported/Conduit-Near-v0-3-0-preview.png" /></a></p>Mmmm now thats pythonic!2006-11-20T22:56:22+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-11-20:2006/11/20/mmmm-now-thats-pythonic/<p>Is it just me or are all the cool apps written in python these days?</p>
<ul>
<li>
<p><a href="http://www.conduit-project.org/">Conduit</a> (hehe thats me!)</p>
</li>
<li>
<p><a href="http://listengnome.free.fr/">Listen</a> (music management and playback)</p>
</li>
<li>
<p><a href="http://www.exaile.org/">Exaile!</a> (amarok styly music app for GTK)</p>
</li>
<li>
<p><a href="http://www.gpixpod.org/wordpress/">GPixPod</a> (manage pictures for your ipod)</p>
</li>
<li>
<p><a href="http://specto.sourceforge.net/">Specto</a> (why procrastinate when you can have an application do it for you)</p>
</li>
<li>
<p><a href="http://www.fluendo.com/elisa/index.php">elisa</a> and pigment (media center by the dudes at <a href="http://www.fluendo.com/">Fluendo</a>)</p>
</li>
<li>
<p><a href="http://www.beatniksoftware.com/gimmie/">Gimmie</a> (desktop revisited)</p>
</li>
<li>
<p><a href="http://raphael.slinckx.net/deskbar/">Deskbar</a> (the most useful app on my desktop)</p>
</li>
</ul>
<p>For extra bonus points I can also freely borrow code from these projects because they are GPL licensed. Yay for FOSS!</p>More pseudoscience..... debunked!2006-11-15T11:23:25+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-11-15:2006/11/15/more-pseudoscience-debunked/<p>Like many people I am becoming incredibly frustrated with the way that science is twisted for political and religions means. It is good to see that this latest pseudoscience article <a href="http://www.telegraph.co.uk/news/main.jhtml?xml=/news/2006/11/05/nosplit/nwarm05.xml">wrongly dismissing climate change</a> from the Sunday Telegraph has been <a href="http://www.guardian.co.uk/commentisfree/story/0,,1947248,00.html">debunked</a>.</p>
<p><em>"[<strong>The Sunday Telegraph Article]</strong> is a dazzling debunking of climate change science. It is also wildly wrong... In keeping with most of the articles about climate change in [the Sunday Telegraph], it is a mixture of cherry-picking, downright misrepresentation, and pseudo-scientific gibberish. But it has the virtue of being incomprehensible to anyone who is not an atmospheric physicist... As for James Hansen, he did not tell the US Congress that temperatures would rise by 0.3C by the end of the past century. He presented three possible scenarios to the US Senate — high, medium, and low. Both the high and low scenarios, he explained, were unlikely to materialise. The middle one was 'the most plausible.' As it happens, the middle scenario was almost exactly right. He did not claim, under any scenario, that sea levels would rise by several feet by 2000."</em></p>
<p>It would be hypocritical to place absolute faith in this one debunking (though I am certain it is an order of magnitude more correct than the original <a href="http://www.telegraph.co.uk/news/main.jhtml?xml=/news/2006/11/05/nosplit/nwarm05.xml">rubbish article</a>). However the lesson is; scientific journals are peer reviewed for a reason, it allows people to have faith that what is published has a certain level of accuracy.</p>
<p>The Sunday Telegraph (a newspaper), by publishing rubbish like this, illustrates the gaping divide between what the general public thinks of as science, and what is actually science. Furthurmore, it is not a leap to assume that if the readers of the newspaper are now so content with the false assumptions they will also lack the necessary critical thinking skills to compare the original Sunday Telegraoh article with the new article debunking the claims. After all, critical thought is hard work for people who believe science articles in newspapers on face value alone.</p>Collection of Thoughts2006-11-13T00:05:20+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-11-13:2006/11/13/collection-of-thoughts/<p>This wont be a particuarly coherant post, More a collection of links that I dont want to lose, and that others might find interesting. Also some updates on the foo that I am working on.</p>
<ul>
<li>
<p>Is anyone offering odds on America going to war with Iran in the next 6 months?</p>
<ul>
<li><a href="http://www.chris-floyd.com/index.php?option=com_content&task=view&id=889&Itemid=135">These</a> <a href="http://www.atlanticfreepress.com/content/view/46/31/">stories</a> make me nervous....</li>
</ul>
</li>
</ul>
<p>One of the great things I like about free software is that there is always something exciting around the corner. Sometimes it just takes way to much procrastinating to find out about it. Here are a few things that I am looking forward to in the next 6 months;</p>
<ul>
<li>
<p><a href="http://lists.freedesktop.org/archives/xorg/2006-September/018263.html">Xorg 7.2 with RandR</a> - Should make managing multiple monitors painless.</p>
</li>
<li>
<p><a href="http://lists.freedesktop.org/archives/xorg/2006-October/019137.html">i810 Modesetting branch</a>. Thanks intel for your rocking OSS drivers.</p>
</li>
<li>
<p>Simple way to <a href="http://article.gmane.org/gmane.comp.gnome.usability/8907">theme colours</a> in GNOME.</p>
</li>
<li>
<p>Some new GNOME infrastructure;</p>
<ul>
<li>
<p><a href="http://mail.gnome.org/archives/gnome-vfs-list/2006-October/msg00008.html">Gnome-vfs replacement</a></p>
</li>
<li>
<p><a href="http://log.emmanuelebassi.net/archives/2006/10/love-and-communication/trackback/">GtkUnique
</a></p>
</li>
</ul>
</li>
<li>
<p><a href="http://lists.ximian.com/pipermail/glade-users/2006-October/003137.html">Glade-3.1</a> (single window mode) - Rocks!</p>
</li>
<li>
<p><a href="http://home.gna.org/nemiver/">Nemiver</a> is awesome, and i'm using it daily.</p>
</li>
<li>
<p>I want to play with <a href="http://www.fluendo.com/elisa/pigment.php">pigment</a>, I particuarly like the idea of having a built in animation framework, making it easy to create fluid user interfaces; like</p>
<ul>
<li>
<p><a href="http://www.podtech.net/scobleshow/technology/1219/the-best-demo-at-web-20-summit-microsofts-photosynth/trackback/">Photosynth</a> (made by microsoft - credit where credit is due).</p>
</li>
<li>
<p><a href="http://www.marceloeduardo.com/blog/2006/11/01/canola-the-mythical-inventor-of-the-harp/">Canola</a> - WOW thats running on a 770!</p>
</li>
<li>
<p><a href="http://weblogs.java.net/blog/joshy/archive/2006/05/aerith_live_fro.html">Aerith</a> - Eeeww thats java (but hot)</p>
</li>
</ul>
</li>
<li>
<p>I have been running <a href="http://www.gnome.org/~jamiemcc/tracker/">Tracker</a> for a long time, and can recommend it to everyone. I would love to see this go into GNOME along with a bung of helper Gtk widgets to make tagging a first class citizen on the GNOME desktop.</p>
</li>
<li>
<p>After the terribly shortsighted Beryl fork its good to see <a href="http://www.go-compiz.org">Compiz get a community website</a>. I wish I had a dual screen graphics card to test all the multihead support stuff that DavidR has just <a href="http://gitweb.freedesktop.org/?p=xorg/app/compiz.git;a=log">added</a> to compiz.</p>
</li>
</ul>
<p>I have also been super busy of late. I have been working hard on <a href="http://www.albatross-uav.org/">Albatross</a>, interfacing with flightgear ad JSBSim, and learning about Eigenstate assignement for controller design. It looks like my next task is to clean up the AVL stuff and start running some code through Matlab to test these theories.</p>
<p>Finally I am back working on <a href="http://www.conduit-project.org/">Conduit</a>, and it looks like its the right time for stuff to come together.</p>
<ul>
<li>
<p>Rewrote the TreeModel stuff to support adding things at run time which is important because....</p>
</li>
<li>
<p>OpenSync just <a href="http://www.opensync.org/wiki/opensync-0.20-notes">released 0.20</a> with better python bindings, so I can start to play with cellphones!</p>
</li>
<li>
<p>I added service discovery of other conduit instances using avahi. This will eventually allow things to be synchronized with remote computers directly and transparently over the local network.</p>
</li>
<li>
<p>Started to add ipod support. When an ipod is plugged in new datasinks should become available which will allow you to synchronize your ipod notes and contacts (and possibly photos thanks to <a href="http://www.gpixpod.org/">GPixPod</a>)</p>
</li>
</ul>
<p>Anyway, thats all for now. Talk to you later or see you at <a href="http://lca2007.linux.org.au/">linux.conf.au.</a></p>Borat2006-10-27T12:36:35+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-10-27:2006/10/27/borat/<p>Words cannot describe how much I am looking forward to the Borat premiere. Before you see the premiere make sure you watch <a href="http://www.webgeordie.co.uk/borat/deletedscenes.htm">these deleted scenes.</a></p>
<p>[youtube]Om7SkkN2T7c[/youtube] </p>Edgy Beta 1 Impressions2006-10-04T13:09:49+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-10-04:2006/10/04/edgy-beta-1-impressions/<p>So I just upgraded my laptop to Ubuntu Edgy Beta 1. This is just a small list of my impressions so far;</p>
<ul>
<li>
<p><a href="https://launchpad.net/distros/ubuntu/+source/usplash/+bug/63875">USplash doesnt work</a> :-(</p>
</li>
<li>
<p>My laptop <a href="https://launchpad.net/distros/ubuntu/+source/linux-source-2.6.17/+bug/62620/">hotkeys do not work</a> anymore (so I cant change screen brightness)</p>
</li>
<li>
<p>The new theme and sound theme are very attractive</p>
</li>
<li>
<p><a href="http://lists.freedesktop.org/archives/compiz/2006-October/000518.html">Compiz-freedesktop 0.2.0</a> works great, thanks to <a href="http://gandalfn.wordpress.com/howto-compiz-aiglx-on-edgy/">this guide </a></p>
</li>
<li>
<p>Boot is way faster, 10 seconds by my stopwatch</p>
</li>
<li>
<p>Suspend and resume is improved</p>
<ul>
<li>
<p>NetworkManager reconnects to networks on resume</p>
</li>
<li>
<p>Compiz works after resume</p>
</li>
</ul>
</li>
<li>
<p>Gnome Power Manager <a href="http://hughsient.livejournal.com/3784.html">battery status graphs rock</a></p>
</li>
</ul>
<p>More news to come...</p>Beryl - The Compiz Fork2006-10-02T16:38:26+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-10-02:2006/10/02/beryl-the-compiz-fork/<p>This post is a little rant on the <a href="http://www.freedesktop.org/wiki/Software/Compiz">compiz</a> fork (<a href="http://www.beryl-project.org/">beryl</a>) and on forks in general.</p>
<p>When I hear people say that "the fork is good because the two goals are different" I bow my head in shame. David R wants the same things as Quinn, a rocking WM/DE.</p>
<p>Its sad that now the fork has "started" there is no going back (in the short term) - the projects will onlycontinue to diverge. Compiz has just gained a whole bunch of features in cvs (pane plugin, multi-head architecture stuff, support for metacity themes) and David has started being more active on the mailing list. Quinn and co. bought this fork on too quickly, sucumbing to every request on the compiz.net forums, and adding poor quality code and hacks all over the show. I mean there has not even been an official compiz release yet!, why fork, how many years did it take to get a solid DE and people are impatient over compiz, a young project with no official release?</p>
<p>If Quinn and co. would have cooled their jets for another month or so then this for wouldnt have happened. If there is a difference in the projects goals it is that one (compiz) has a BDFL that has a grander vision and quality for the project, and that the other (beryl) doesnt. Its a free for all.</p>
<p>About the GNOME dependancies - its only gconf. Quinn wrote a plugin to replace gconf (csm) but it was done in such a hackish way that it wasnt clean. I wish they would have just improved the csm plugin to the quality that it could be accepted upstream and then this whole fork rubbish could have been avoided.</p>
<p>If you read <a href="http://lists.freedesktop.org/archives/compiz/2006-September/000487.html">Davids latest comments</a> you will see that he is concerned that the current changes in beryl (and by extrapolation most future changes) will be done in such a (hackish?) way to make them incompatible with upstream compiz.</p>
<p>Many of the plugins in beryl touch the core and are no longer implemented in a plugin type way. David offered an invitation for suggestions on how to improve the core APIs so that these beryl features could be implemented as plugins in a nicer way but no-one has stepped up and made said suggestions.</p>
<p>I guess its just easier to fork than it is to talk to a person these days.....</p>
<p>In essence beryl will be take take take and no give</p>
<p>That makes me sad</p>Conduit and Other News2006-09-25T21:52:49+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-09-25:2006/09/25/conduit-and-other-news/<p>So I finally got around to releasing <a href="http://www.conduit-project.org/">Conduit 0.2.0</a>, which, while having many limitations, is relatively useful and safe to use on a daily basis. I have been swaped by university work at the moment, as this is the last week, but I hope to be back developing again soon.</p>
<p>Other Things on the hack-agenda over the next few weeks (months)</p>
<ul>
<li>
<p><a href="http://www.albatross-uav.org/">Albatross</a> work</p>
</li>
<li>
<p><a href="http://www.conduit-project.org/wiki/TODO">Refactoring Conduit</a> to be more MVC</p>
</li>
<li>
<p>Thinking about synchronization as it can be applied to <a href="http://www.conduit-project.org/wiki/SynchronizationService">all of GNOME</a></p>
</li>
<li>
<p>Write some GTK widgets for <a href="http://www.gnome.org/~jamiemcc/tracker/">Tracker</a></p>
</li>
</ul>Conduit v0.2.0 Coming Soon2006-09-11T21:31:56+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-09-11:2006/09/11/conduit-v020-coming-soon/<p>I have been working really hard for the last month on getting the <a href="http://www.conduit-project.org/">Conduit</a> core to a v0.2.0 ready state.... However after talking to a few people I have decided to delay v0.2.0 by a week to give more time for testing, and to increase the usefullness of some of the included dataproviders (Flickr and Fspot in particular).</p>
<p>The rationale behind this is, however cool an Idea Conduit is, it is only as useful as the things it can be used to sync with. I want this release to make a big splash, so I want the ability to sync with included GNOME apps to work really well. This means focusing a lot of testing on the Tomboy and F-spot datasources, and the Flickr datasink.</p>
<p>I encourage users to check out the latest version from svn to help in testing. In the meantime, here is a screenshot of Conduit running happily on Edgy.</p>
<p><a href="http://www.johnstowers.co.nz/blog/static/images/imported/Edgy.png"><img alt="Conduit running on Ubuntu Edgy" src="http://www.johnstowers.co.nz/blog/static/images/imported/Edgy.png" /></a></p>Conduit 0.1 Released2006-08-09T16:43:50+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-08-09:2006/08/09/conduit-01-released/<p>After a solid month of work I just release v0.1 of <a href="http://www.conduit-project.org/">Conduit.</a></p>
<p><a href="http://www.johnstowers.co.nz/blog/index.php/2006/08/09/conduit-01-released/conduit-01-screenshot/"></a></p>
<p><a href="http://www.johnstowers.co.nz/blog/index.php/2006/08/09/conduit-01-released/conduit-01-screenshot/"><img alt="Conduit 0.1 Screenshot" src="http://johnstowers.co.nz/blog/wp-content/uploads/2006/08/Screenshot-Conduit.png" /></a></p>
<p>Conduit is a synchronization solution for <a href="http://www.gnome.org/">GNOME</a> which allows the user to take their emails, files, bookmarks, and any other type of personal information and synchronize that data with another computer, an online service, or even another electronic device.</p>
<p>This version is the first version which is actually usable so I thought I would release it to gather feedback, particuarly about the GUI, and how people interact with the application.</p>
<p>The next version will bring more bling including</p>
<ul>
<li>
<p>Treeview for datasources and sinks. They will be categorised into groups to more easily distinguish between them</p>
</li>
<li>
<p>Threading cleanups so that sync's may be cancelled while in process</p>
</li>
<li>
<p>Flickr datasource and sink</p>
</li>
<li>
<p>Two way sync for backpackit.com and files</p>
</li>
<li>
<p>Gmail dataprovider tidy ups</p>
</li>
</ul>
<p>I am looking for help and interested developers. Check it out!</p>
<p><a href="http://www.conduit-project.org/">http://www.conduit-project.org/</a></p>Epiphany just got a whole bunch better2006-07-30T15:03:36+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-07-30:2006/07/30/epiphany-just-got-a-whole-bunch-better/<p>I just stumbled across <a href="http://www.sstuhr.dk/epiphany-extensions/">the unoficial epiphany extensions site.</a> I would recommend all users of epiphany install at least the;</p>
<ul>
<li>
<p>Undo close tab extension</p>
</li>
<li>
<p>Only one close button extension. This also auto-resizes the tab width!</p>
</li>
<li>
<p>Middle click tab close extension. I have eliminated the scroll wheel from my life to combat RSI so with this (and the push scroll extension) I can operate epiphany using the middle mouse button.</p>
</li>
</ul>
<p>See how attractive epiphany looks now?</p>
<p><img alt="Epiphany hotness" src="http://johnstowers.co.nz/blog/wp-content/uploads/2006/07/Screenshot-Unofficial%20Epiphany%20Extensions.png" /></p>Matlab and Simulink on Dapper2006-05-11T16:47:50+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-05-11:2006/05/11/matlab-and-simulink-on-dapper/<p>Note to self: If you need to unstall simulink on ubuntu dapper then make sure you install libxft1</p>GNOME on the horizon2006-04-26T21:38:48+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-04-26:2006/04/26/gnome-on-the-horizon/<p>My responses to <a href="http://www.pthree.org/2006/04/25/gnome-integration/">this</a> well composed post of GNOME constructive criticism.</p>
<p>Interesting and well written post. I am also not a gnome expert, only a user and part time developer. Here are some suggestions regarding each of the points you made;</p>
<p><em>Firstly a few general comments.</em>
Personally on Ubuntu Dapper Beta, Gnome is considerably faster than KDE in my experience, but this may just be a variation of the placebo effect!. Either way all the recent GNOME performance work has sped it up considerably since the version in Breezy!</p>
<p>Secondly I think a number of your points make the assumption that GNOME = Nautilus, or GNOME = Nautilus + Metacity. This is a common belief (or if indeed this is what you though) and incorrect. HOWEVER, this is why we must admit that proportionally a lot of polish must go into these two applications. I dont think that throwing features at Nautilus is the answer, it will also never be the GNOME answer, and I am thankful for it. But you do raise some good points, and ignoring any GNOME user is the wrong thing to do.</p>
<p>Now, point by point responses</p>
<p><em>Instant Messaging:</em>
Check out the <a href="http://www.galago-project.org/about.php">galago desktop presence</a> framework. While still on the horizon I forsee some form of integration between the entire desktop, gaim, and this as the glue</p>
<p><em>Voice Over IP:</em>
Check out <a href="http://ekiga.org/">ekiga</a>. Maybe at some stage this may make it into core GNOME. Either way there is possiblity for integration with above.</p>
<p><em>File Sharing:</em>
I could not agree more. Apparently nautilus (gnome-vfs) supports rendevous discovery of shares and the like but I dont think this is fully taken advantage of (or I havent seen it in GNOME 2.14). Perhaps something could be done with <a href="http://www.0x50.org/">cherokee web server</a> + WebDAV + automatic nautilus discovery + gnome-vfs WebDAV + places sidebar. Edit- This was done <a href="http://elysium-project.sourceforge.net/epittance/">here</a>, but i think the project died.</p>
<p><em>Secure Shell GUI:</em>
This is already there. Check out Places -> Connect To Server -> SSH. I presume you can also enter sftp://user@server/path</p>
<p><em>More Command Line Tools:</em>
While I understand the problem you want to solve; easier access to command line. I dont think shoving everything into the file browser is the right way to go about it. If GNOME can think of a better way to interact with the command line in a modern GUI environment then this is a step in the right direct. Putting the terminal into nautilus I dont think so much.</p>
<p><em>Menu Editing:</em>
Ubuntu Dapper ships the alacarte menu editor. It has also been <a href="http://mail.gnome.org/archives/desktop-devel-list/2006-April/msg00305.html">proposed for inclusion</a> into GNOME 2.16. Its a step in the right direction.</p>
<p><em>Eye Candy:</em>
Do NOT open this can of worms. Personally I think the GNOME has KDE beat when it comes to eye candy. The eyecandy discussion will never be resolved. Its all a matter of preference, I like toyota, you like mazda etc. At the framework GNOME can draw nice graphics using Cairo, and fun things are happening FOR EVERYONE IN THE LINUX COMMUNITY, GNOME AND KDE using <a href="http://www.linuxedge.org/?q=node/58">compiz and XGL</a> or AIGLX. You do have some points, there are features coming into GTK to be able to detect if a compositing manager is running, but if I want GNOME to keep looking prettier than KDE (joke) then <a href="http://macslow.thepimp.net/?p=40">stuff needs to go into GTK to make this easier!</a></p>
<p><strong>Conclusion</strong>
I think that this is a very interesting time for GNOME. There is all this cool framework level stuff sitting there, simmering away, ready to be cooked into GNOME in some amazing ways. <a href="http://www.chipx86.com/wiki/Leaftag">leaftag</a>, <a href="http://galago.sf.net/">galago</a>, <a href="http://browserbookapp.sourceforge.net/deskbar-2-14-screencast.html">deskbar</a> (and the buzz its generating), <a href="http://beaglewiki.org/Main_Page">desktop</a> <a href="http://www.advogato.org/person/jamiemcc/diary.html?start=4">search</a>, <a href="http://www.gnome.org/~davyd/gnome-2-14/">performance improvements</a>, <a href="http://www.linuxedge.org/?q=node/58">compiz and XGL</a>, and <a href="http://www.beatniksoftware.com/blog/index.php?p=38">MUCH</a> <a href="http://www.beatniksoftware.com/blog/index.php?p=38">MORE</a>.</p>
<p>Comments welcome</p>Motivation....2006-04-23T22:06:09+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-04-23:2006/04/23/motivation/<p>"Nobody should start to undertake a large project. You start with a small <em>trivial</em> project, and you should never expect it to get large. If you do, you'll just overdesign and generally think it is more important than it likely is at that stage. Or worse, you might be scared away by the sheer size of the work you envision.
So start small, and think about the details. Don't think about some big picture and fancy design. If it doesn't solve some fairly immediate need, it's almost certainly over-designed. And don't expect people to jump in and help you. That's not how these things work. You need to get something half-way <em>useful</em> first, and then others will say "hey, that <em>almost</em> works for me", and they'll get involved in the project. "</p>
<p>Linus Torvalds</p>Nuclear Energy2006-04-18T13:43:47+02:00John Stowerstag:www.johnstowers.co.nz/blog,2006-04-18:2006/04/18/nuclear-energy/<p>It is good to see that <a href="http://www.washingtonpost.com/wp-dyn/content/article/2006/04/14/AR2006041401209.html">some environmentalists</a> are not as dogmatic in their anti-nuclear views as others. I think it is high time that New Zealand had a mature and open nuclear energy debate.</p>Epiphany is AWESOME!2006-03-15T21:55:45+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-03-15:2006/03/15/epiphany-is-awesome/<p>After reading <a href="http://ploum.frimouvy.org/?2006/03/15/100-why-you-should-try-epiphany-as-your-default-browser-with-gnome-214">this post</a> I have now switched from firefox to epiphany. I would have done so earlier if this was hyped during the (or <a href="http://davyd.livejournal.com/167967.html">Davyd's</a>) 2.14 release notes!</p>
<p>My favorite features (so far) are</p>
<ul>
<li>
<p>Extensions (especially integration with liferea)</p>
</li>
<li>
<p>When a page cannot epiphany offers the GCace or Archive.org version</p>
</li>
<li>
<p>Starts faster</p>
</li>
<li>
<p>Taggable style bookmarks!</p>
</li>
<li>
<p>Deskbar integration</p>
</li>
<li>
<p>Download manager (with notification area icon)</p>
</li>
</ul>
<p>I would recommend any gnomie out there to switch to epiphany today!</p>Ubuntu Dapper Flight52006-03-13T15:33:05+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-03-13:2006/03/13/ubuntu-dapper-flight5/<p>Just upgraded to flight 5 and feeling quite positive about the state of ubuntu. Suspend is working agin on my laptop, and so is the SD card slot. I have included a screenshot below.
<a href="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/DapperF5.png"></a></p>
<p><a href="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/DapperF5.png"><img alt="Screenshot of Dapper Flight 5" src="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/DapperF5.png" /></a></p>
<p>Also I plan to do a review of my laptop at some stage in the near future (Panasonic CF-R4).</p>Updated XGL and Compiz2006-03-12T00:22:47+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-03-12:2006/03/12/updated-xgl-and-compiz/<p>With the great help of those on the Ubuntu forums XGL/Compiz is looking greater than ever. The <a href="http://www.ubuntuforums.org/showthread.php?t=139265">Updated Compiz</a> adds the opacity plugin and darkens the background for the expose effects. But better than that, the <a href="http://www.ubuntuforums.org/showpost.php?p=798732&postcount=1">Updated XGL</a> removes all the flicker that I was seeing.</p>Kororaa XGL Live CD2006-03-09T20:58:36+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-03-09:2006/03/09/kororaa-xgl-live-cd/<p>Well I have just tried the Kororaa live cd on my P4 2.4Ghz with Nvidia FX5200 and it works flawlessly. Xgl and Compiz are amazing!</p>
<p>I only hope that the recent news that Compiz will soon run on Xgl AND aiglx doesn't spell the end (or slow painful death) or metacity as the gnome window manager. If this business has shown me anything its that David (xgl and compiz author) is a man machine and can accomplish anything!</p>XGL on i9152006-03-07T08:02:02+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-03-07:2006/03/07/xgl-on-i915/<p>Well, all the cool kids are doing it so I thought that I better have a go. XGL and compiz run on my integrated i915 video without problems and I am amazed at the speed and quality for pre-alpha software.</p>
<p><a href="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/XGL.png"><img alt="XGL" src="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/XGL.thumbnail.png" /></a> <a href="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/XGL2.png"><img alt="XGL2" src="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/XGL2.thumbnail.png" /></a> <a href="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/XGL3.png"><img alt="XGL3" src="http://johnstowers.co.nz/blog/wp-content/uploads/2006/03/XGL3.thumbnail.png" /></a></p>
<p>After all of the hassle of building Compiz and friends (excluding XGL) from CVS I found <a href="http://www.ubuntuforums.org/showthread.php?t=139265">this</a> and <a href="http://ubuntuforums.org/attachment.php?attachmentid=6675&d=1141431998">this (includes new opacity plugin).</a><a href="http://ubuntuforums.org/attachment.php?attachmentid=6675&d=1141431998">
</a></p>Gnome Documentation System2006-03-02T00:27:01+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-03-02:2006/03/02/gnome-documentation-system/<p>Just started hacking on my attempt to implement some sort of more modern documentation system for the gnome user guide.</p>
<p>So far this involves a few components; a modified version of moinmoin 1.5, a hacked version of yelp with xml-rpc support, and some python scripts.</p>
<p>Basically the idea is that moinmoin now supports docbook generation and has a nice xml-rpc interface. So the idea is that periodically the wiki version of the gnome-user-guide is crawled and docbook is generated. Also, yelp can fetch the latest version of the documentation from the wiki and display it.</p>
<p>A bit of work to go but a nice project covering lots of disiplines</p>Gnome Documentation2006-02-20T01:08:19+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-02-20:2006/02/20/gnome-documentation/<p>Well by virtue of having a working jhbuild install of Gnome 2.13.91 I have volunteered myself to update the nautilus section of the gnome user docs. Well what a Job that is shaping up to be!</p>
<p>I have been rewriting a lot of the docs and taking a heap of (matching) screen shots along the way so when I finish (which will be in a day or two) the new user guide will be rocking!</p>Successful build of Gnome 2.13.91 using jhbuild2006-02-18T01:02:53+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-02-18:2006/02/18/successful-build-of-gnome-21391-using-jhbuild/<p>Well after a little bit of fluffing around, but an otherwise streamlined build process I got a installation of Gnome 2.13.91 going. Thanks a lot to the following links</p>
<p>https://wiki.ubuntu.com/LiveCDCustomizationHowTo
http://live.gnome.org/GnomeLiveCd_2fHowTo</p>
<p><a href="http://johnstowers.co.nz/blog/wp-content/uploads/2006/02/2.13.91_Screenshot.png"><img alt="2.13.91 Desktop" src="http://johnstowers.co.nz/blog/wp-content/uploads/2006/02/2.13.91_Screenshot.png" /></a></p>
<p>The first thing I notice about the new version: fast! Nautilus and startup have become a lot faster. I really like the eog simplification that has gone on, and the fusa stuff seems to work really well. I will go and file a few bugs about the operation of nautilus and the blinging new search feature.
Now I can move onto the next step of my "automated livecd using jhbuild master plan"</p>Gnome 2.13.91 jhbuild Fun2006-02-17T02:17:17+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-02-17:2006/02/17/gnome-21391-jhbuild-fun/<p>Well I spent most of the afternoon trying to get Gnome 2.13.91 (2.14 Beta 2) to build. The first problem was with libxklavier. The version in jhbuild fails to build, I filed this <a href="http://bugzilla.gnome.org/show_bug.cgi?id=331413">bug</a> but in the meantime replacing with libxklavier2.1 fixes it.</p>
<p>The only other problem is with epiphany which fails to build. It cannot find iso-codes. I actually think the problem is with iso-codes because running in jhbuild shell, it cannot be found. Maybe iso-codes make install is broken.</p>
<p>Anyway tomorrow I will have a go at building jhbuild in chroot (im trying to get a working GnomeLiveCD from using jhbuild). More on that later</p>Mono and jhbuild2006-02-16T02:23:31+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-02-16:2006/02/16/test/<p>Well today I spent time trying to package up a jhbuild moduleset for mono. I got a bit distracted and wrote a helper script, when given the mono base url, spiders the download page for tar.gz releases, downloads and md5sums them, and updates the moduleset accordingly. Hopefully this will make it easier to keep the moduleset update because the script should do all the work.</p>
<p>Anyway it was fun to do a bit in python with xml.dom</p>Tango Icon Internals and Statistics2006-02-15T02:12:08+01:00John Stowerstag:www.johnstowers.co.nz/blog,2006-02-15:2006/02/15/tango-icon-internals-and-statistics/<p>So I have been trying to think of a way to change the appearance of all the tango icons in one go. I envision being able to do the following- "Hmm I dont like tango, its a bit blue. What if I replace all the tango blue colours with lighter ones". My understanding is that SVG (or maybe inkscapes's implementation, im not sure) cannot define external stylesheets or palettes for svg files, so if you want to change something in lots of svg files en-masse then you have to grep/sed it.</p>
<p>So my first investigation was to find out how may colours esist in the tango icon theme (remember that the icon palette supplied has just 27 colours)</p>
<p>john@nzjrs-desktop:~/src/tango$ find /home/john/src/tango/tango-icon-theme -name "<em>.svg" -exec egrep -o 'fill:#[a-f0-9]{6,6}' {} \; | sort | uniq | wc -l
151
john@nzjrs-desktop:~/src/tango$ find /home/john/src/tango/tango-icon-theme -name "</em>.svg" -exec egrep -o '#[a-f0-9]{6,6}' {} \; | sort | uniq | wc -l
749</p>
<p>Hmm, this is going to be harder than I thought!</p>