tag:blogger.com,1999:blog-25907862435145858952024-03-13T15:11:48.928+00:00Camltastic!Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.comBlogger43125tag:blogger.com,1999:blog-2590786243514585895.post-90605496352853030872009-03-05T18:16:00.002+00:002009-03-05T18:17:00.169+00:00New blogI finally got sick of blogspot's inability to handle basic editing tasks, and moved to a <a href="http://rwmj.wordpress.com/">new blog on wordpress</a>.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-87323131711484452922009-03-04T18:42:00.006+00:002009-03-04T18:50:16.568+00:00Talk: Using the Fedora Windows cross-compiler<table><tr><td>Date:</td><td><b>Sunday, 8th March 2009</b></td></tr><br /><tr><td>Time:</td><td> <b><a title="Times at other locations around the world" href="http://www.timeanddate.com/worldclock/fixedtime.html?day=8&month=3&year=2009&hour=18&min=15&sec=0&p1=0">18:15 UTC</a></b></td></tr><br /><tr><td>Location:</td><td> <b>Virtual -- #fedora-classroom on <a href="https://fedoraproject.org/wiki/Communicate/IRCHowTo">irc.freenode.net</a></b></td></tr><br /><tr><td>Details:</td><td> <a href="https://fedoraproject.org/wiki/Classroom#The_Current_Timeline">https://fedoraproject.org/wiki/Classroom#The_Current_Timeline</a></td></tr><br /><tr><td>Background:</td><td> <a href="https://fedoraproject.org/wiki/Features/Windows_cross_compiler">https://fedoraproject.org/wiki/Features/Windows_cross_compiler</a></td><br /></tr></table><br />A talk and interactive session, <b>"Using the Windows cross-compiler"</b>:<br /><ul><li> API basics: POSIX, libc, Win32, gtk, Qt, etc.</li><br /><li> Cross-compiler basics</li><br /><li> Practical demonstration:<br /> setting up the cross-compiler in Fedora<br /> compiling a small Gtk program<br /> testing it in Wine<br /> building a Windows installer</li><br /><li> Future directions (Win64, Mac OS X ?)</li><br /><li> How to get involved</li></ul><br />If you have root access to a Fedora 10 or rawhide install (i386 or x86-64 only) you can follow along with the practical part.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-88030646712191451962009-02-06T20:41:00.005+00:002009-02-06T22:47:32.861+00:00Photos from OCaml Users Meeting, Grenoble, 2009We had a very successful <a href="http://cocan.org/events/europe/ocamlmeetinggrenoble2009">OCaml User Meeting</a> this week, with 45 people coming from around Europe to Grenoble.<br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-people.jpg"><br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-xl.jpg"> Xavier Leroy on progress made over 2008<br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-sg.jpg"> Sylvain Le Gall <a href="http://www.ocamlcore.org/">OCamlCore.org</a> and OCaml as fast as C<br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-mg.jpg"> Maxence Guesdon <a href="http://home.gna.org/cameleon/chamo.en.html">Chamo and Cameleon</a><br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-fo.jpg"> Florent Ouchet <a href="http://forge.ocamlcore.org/docman/view.php/77/34/VSYML-ocaml-meeting-2009.pdf">VHDL symbolic simulation</a><br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-dt.jpg"> David Teller <a href="http://batteries.forge.ocamlcore.org/">Batteries Included</a><br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-ct.jpg"> Christophe Troestler <a href="http://pa-do.forge.ocamlcore.org/">pa_do (delimited overloading) syntax</a><br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-rj.jpg"> Richard Jones (me) <a href="http://fedoraproject.org/wiki/MinGW">OCaml Windows cross-compiler</a><br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-cr.jpg"> Christophe Raffalli <a href="http://dypgen.free.fr/">Parsing with dypgen</a><br /><br />Xavier Leroy on the right:<br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-group.jpg"><br /><br />Dinner the evening before:<br /><br /><img src="http://www.annexia.org/tmp/caml2009/cm-dinner.jpg">Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com3tag:blogger.com,1999:blog-2590786243514585895.post-90421813945253012912009-01-29T19:32:00.003+00:002009-01-29T19:36:37.935+00:00Talk in London on Fedora & Windows cross-compilationI am giving a talk and demonstration <a href="http://www.gllug.org.uk/index.php?/archives/44-31st-January-Greater-London-LUG-Meeting.html">in London (UK) this Saturday afternoon, 31st January</a> on the subject of the <a href="https://fedoraproject.org/wiki/MinGW">Fedora MinGW (Windows cross-compiler) project</a>. Free entry, everyone is invited!<br /><br /><a href="http://camltastic.blogspot.com/search/label/mingw">Previous MinGW postings on this blog ...</a>Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-74544207472722204272009-01-10T10:03:00.007+00:002009-01-10T18:03:52.583+00:00ocsigen (OCaml web framework) benchmarked<a href="http://eigenclass.org/R2/writings/standalone-ocaml-webapps">This benchmark compares the OCaml web framework ocsigen against Ruby on Rails and lighttpd+C</a> with some excellent results for the OCaml framework:<br /><table style="width:40em; background-color: #ffeeee;"><tr><td></td><th> Reqs/sec</th><th> Mem usage</th></tr><tr><td>Rails with mongrel, 1 process </td><td> 260</td><td> 49MB</tr><tr><td>Rails with mongrel via nginx (rev proxy), 1 proc </td><td>220</td><td> ~51MB</tr><tr><td>Rails with mongrel, 4 processes via nginx </td><td>430</td><td> ~200MB</tr><tr style="background-color:#ff8888; font-weight: bold;"><td>OCaml ocsigen (1 process)</td><td> 5800</td><td> 4.5MB</tr><tr><td>lighttpd with FastCGI app in C, 20 procs </td><td> 9300 </td><td> 4.5MB</tr></table><br />OCaml is not just an order of magnitude faster and an order of magnitude more memory efficient, but it also provides complete compile-time safety, catching multiple errors at compile time which would otherwise only show up after extensive testing.<br /><br />More discussion on <a href="http://www.reddit.com/r/programming/comments/7oif5/standalone_web_applications_using_ocaml_ocsigen/">this reddit thread</a>.<br /><br />There's a Fedora ocsigen package <a href="https://bugzilla.redhat.com/show_bug.cgi?id=460894">waiting for review here</a>.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-614073273572889822009-01-02T20:49:00.004+00:002009-01-02T21:02:02.729+00:00Home server, part 4, installing the OSFor want of a cable, my <a href="http://camltastic.blogspot.com/2008/09/home-server-part-3-quick-update.html">home file server</a> wasn't coming along very well, but today I hooked up the <a href="http://www.maplin.co.uk/module.aspx?moduleno=28724">2.5" IDE to PATA converter cable from Maplin</a> with the hard drive from the disassembled Viglen MPC-L:<br /><img src="http://annexia.org/tmp/hs-disk.jpg"><br /><br />The Viglen was really easy to disassemble by the way. Two screws on the back hold on the backplate, and then the entire motherboard/hard disk assembly slides straight out. Another three screws let you remove the hard drive.<br /><br />I settled on relatively simple route to install CentOS. I used <a href="http://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine">Red Hat's KVM virtualization</a> to run a VM, attaching the physical hard drive and the (virtual) CentOS DVD ISO. It sounds complicated, but all you need is this <a href="http://libvirt.org/">virt-install</a> command line to do it (the host is Fedora 10):<pre><br />virt-install --connect=qemu:///system \<br /> -n centos5 -r 512 \<br /> -v --accelerate \<br /> -c /root/CentOS-5.2-i386-bin-DVD.iso \<br /> -f /dev/sde \<br /> --vnc --vncport=5900</pre><br />(Adjust the path to the CentOS DVD ISO, and the physical hard drive device as appropriate).<br /><br />CentOS 5.2 installed in about 15 minutes:<br /><img src="http://annexia.org/tmp/centos-install.png">Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-12898177210464520572008-12-30T13:29:00.006+00:002008-12-30T14:27:37.213+00:00Destroying old hard drivesThis is a simple, cheap method for destroying old hard drives, making the data unrecoverable against casual attackers and identity fraudsters (although probably not some hypothetical government agency with multi-million dollar resources).<br /><br />For this you will need a stack of old hard drives:<br /><img src="http://www.annexia.org/tmp/1-hdds.jpg"><br /><br />An electric drill with a <a href="https://secure.wikimedia.org/wikipedia/en/wiki/Drill_bit#Twist_drill">twist drill bit</a> (suitable for going through metal), and most importantly some eye protection:<br /><img src="http://www.annexia.org/tmp/2-need.jpg"><br /><br />Line up the hard drives against the wall and drill straight through them. I didn't show it in this picture, but in fact I drilled through from the other (PCB) side to ensure that I went through the PCB but didn't go through any components that might explode:<br /><img src="http://www.annexia.org/tmp/3-wall.jpg"><br /><br />Sunlight where there's not supposed to be sunlight!<br /><img src="http://www.annexia.org/tmp/4-sunlight.jpg"><br /><br />For a few of the drives, mainly older ones, I couldn't get all the way through, but I got through to the platters, which is the important part:<br /><img src="http://www.annexia.org/tmp/5-notthru.jpg"><br /><br />Now you can see why eye protection is <b>not</b> optional. This old IBM SCSI-LVD drive had glass platters which shattered into tiny, sharp shards of metal-plated glass when the drill went through:<br /><img src="http://www.annexia.org/tmp/6-shattered.jpg"><br /><br />For extra assurance, I will soak the drives in a bucket of water for a few days before disposing of them:<br /><img src="http://www.annexia.org/tmp/7-water.jpg">Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com8tag:blogger.com,1999:blog-2590786243514585895.post-82424813443429024272008-12-08T18:57:00.005+00:002008-12-08T19:12:28.678+00:00Stop spying on my encyclopedia readingRecently the unaccountable UK "Internet Watch Foundation" added <a href="http://www.theregister.co.uk/2008/12/07/brit_isps_censor_wikipedia/">pages from Wikipedia to a secret list of censored pages</a>.<br /><br />I would like to make the point that <b>no one should be prosecuted for reading an encyclopedia</b>. Furthermore, no free, democratic society should tolerate authorities spying on people reading works of knowledge.<br /><br />Let's together stop this spying now.<br /><br /><b>Here is a simple action that you can take right now</b>, that won't cost you any time or money. When linking to any page on Wikipedia, use the secure URL:<pre><big><a href="https://secure.wikimedia.org/wikipedia/en/wiki/Main_Page">https://secure.wikimedia.org/wikipedia/en/wiki/Main_Page</a></big></pre>(Replace <code>Main_Page</code> with the name of the Wikipedia page as usual. You can also replace the language (<code>en</code>) or link to <code><a href="https://secure.wikimedia.org/wikipedia/commons/wiki/Image:Bundesarchiv_Bild134-B0335,_Fregatte_%22Thetis%22_vor_Anker.jpg">commons pages</a></code>).Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com4tag:blogger.com,1999:blog-2590786243514585895.post-67355183774847298772008-12-07T18:00:00.002+00:002008-12-07T18:00:00.962+00:00Fedora Rawhide has OCaml 3.11.0<a href="http://fedoraproject.org/wiki/Releases/Rawhide">Fedora Rawhide</a> [the experimental/development version of Fedora] has been completely rebuilt with OCaml 3.11.0, and all library and application problems that were found have been patched.<br /><br />List of packages: <a href="http://cocan.org/fedora#Package_status">http://cocan.org/fedora#Package_status</a><br /><br /><a href="http://caml.inria.fr/pub/ml-archives/caml-list/2008/12/553a7160e318a589d5fb1fb53c7039de.en.html">Mailing list announcement</a>Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-14219427079958317532008-12-07T12:09:00.005+00:002008-12-07T12:44:35.522+00:00Slashdot groupthink<a href="http://slashdot.org/comments.pl?sid=1053709&threshold=-1&commentsort=0&mode=nested&cid=26016165">This may be the first time a comment of mine has been modded down to -1</a> on Slashdot. I'm questioning whether the inefficiency of glib outweighs the speed advantage of C. Very few of the replies get it. Perhaps this proves the people only read the first sentence of any posting ... tl;dr.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com2tag:blogger.com,1999:blog-2590786243514585895.post-55817069274355180082008-11-26T11:39:00.003+00:002008-11-26T11:42:59.350+00:00Fedora 10 & OCamlYou can join in the <a href="http://planet.fedoraproject.org/">general Fedora 10 fun</a> here, but a quick note that Fedora 10 comes with stable OCaml 3.10.2 and <a href="http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora">68 OCaml packages</a>, making us the fastest, best supported functional language in Fedora.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-32105884606516123722008-11-24T16:44:00.003+00:002008-11-24T17:28:53.165+00:00Common mistakes cross-compiling MinGW packages<h4>Using the headers from /usr/include</h4>The headers in <code>/usr/include</code> are for the native libraries installed on the system, and it's highly unlikely they will work for cross-compilation. By "won't work" I mean that types and structure fields could be different, resulting in a segfault.<br /><br />The Fedora MinGW project takes two steps to avoid using native libraries by accident: Firstly GCC is configured so it looks in <code>/usr/i686-pc-mingw32/sys-root/mingw/include</code> and <i>never</i> looks in <code>/usr/include</code> (as long as you don't tell it to). Secondly we supply a replacement <code><a href="http://hg.et.redhat.com/misc/fedora-mingw--devel/?f=99aa09eb6e55;file=example/mingw32-example.spec">%{_mingw32_configure}</a></code> RPM macro which sets <code>PKG_CONFIG_PATH</code>, so any pkg-config done will pick up the cross-compiled libraries' configuration instead of any native libraries' configuration.<pre><br />$ PKG_CONFIG_PATH=/usr/i686-pc-mingw32/sys-root/mingw/lib/pkgconfig \<br /> pkg-config --cflags glib-2.0<br />-mms-bitfields -I/usr/i686-pc-mingw32/sys-root/mingw/include/glib-2.0<br />-I/usr/i686-pc-mingw32/sys-root/mingw/lib/glib-2.0/include <br /></pre><br />One thing that can still go wrong is that you don't have the cross-compiled library installed and it then picks up the native library. For example, you missed a BuildRequires line. That mistake usually becomes evident when the program tries to link, because linking a cross-compiled Windows binary to a native Fedora library won't work.<br /><br /><h4>Not setting --prefix</h4>You likely don't want to install Windows binaries and libraries under <code>/usr</code> or <code>/usr/local</code>. For a start it's better to keep Windows things in one place, and the <a href="https://fedoraproject.org/wiki/Packaging/MinGW#Filesystem_layout">packaging guidelines have specified that place</a> to be <code>/usr/i686-pc-mingw32/sys-root/mingw</code>. But mainly it's not a good idea to mix up native and cross-compiled libraries, which will cause all sorts of problems as in the point above.<br /><br />If you use <code>%{_mingw32_configure}</code> in RPM specfiles, or the <code>mingw32-configure</code> command, then paths will be set correctly for you.<br /><br /><h4>Not using a portability library</h4>If you're writing the program yourself, or if you're doing the often difficult work of <a href="http://camltastic.blogspot.com/2008/10/mingw-compile-software-for-windows.html">porting an existing application</a>, use a portability library to help you. Which you choose is up to you and depends on many factors, but we would recommend that you look at these ones:<ul><li> <a href="http://www.gtk.org/">glib2 and gtk2</a> </li><li> <a href="http://www.gnu.org/software/gnulib/">Gnulib</a> </li><li> <a href="http://apr.apache.org/">apr</a> </li><li> <a href="http://trolltech.com/">QtCore</a>, <a href="http://www.boost.org/">Boost</a>, <a href="http://pocoproject.org/">POCO</a> or <a href="http://www.mozilla.org/projects/nspr/">NSPR</a> if you're using C++ </li></ul><br /><br /><h4>Writing your own build system</h4>While it's fashionable to dislike <code>autoconf</code> and m4 macros, it is still by far the easiest way to both build your program on multiple systems, and to cross-compile. So use autotools or <a href="http://www.cmake.org/">cmake</a>, and definitely don't write your own build system. Discourage other projects from writing their own build systems too.<br /><br />This really comes down to bitter experience. Every project we have had to port that has used its own build system has been far more of a headache than those that just used autoconf or cmake.<br /><br /><h4>Running programs during the build process</h4>When cross-compiling, it's always a mistake to run programs during essential build steps. The problem is that you can't be sure that binaries can be made to work in the build environment. For Windows binaries, there is some chance of running them under <a href="http://www.winehq.org/">Wine</a>, but Wine itself is incompatible with autobuild environments like <a href="http://fedoraproject.org/wiki/Projects/Mock">mock</a> and <a href="http://koji.fedoraproject.org/koji/">Koji</a>. Furthermore Wine only works on x86 platforms, and it's not possible to use it at all when cross-compiling from other architectures like PPC.<br /><br />Running programs during <code>make test</code> is normal and useful though.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-62668427927144143832008-11-19T20:34:00.008+00:002008-11-24T16:44:24.144+00:00Egg & "Verified by Visa"Message sent to <a href="http://egg.com/">Egg</a> today about <a href="http://www.theregister.co.uk/2008/10/23/vbyv_analysis/">Verified by Visa</a>:<blockquote><br />Dear Sir/Madam,<br /><br />I would like to permanently opt out of "Verified By Visa" when making purchases online. It just moves the liability on to me and the technical implementation of it is frankly crap. If not, I'll cancel my card (I expect you'll be <a href="http://news.bbc.co.uk/1/hi/business/7224268.stm">happy</a> about that) since it's no longer useful for purchases.<br /><br />If however you are going to introduce some scheme which is really secure, such as a hardware token or one-time credit card numbers or authorization by SMS message, then let me know.<br /></blockquote><br /><br />Update (2008-11-20) — a dull form reply from Egg:<blockquote><br />The Secure online code service is supported by Verified by Visa and MasterCard Secure Code. It protects your card with a password, giving you added security when you shop online.<br /> <br />When you make purchases online with participating retailers, you'll be presented with a receipt at the end of the checkout process. The receipt includes details of your purchase, showing retailer name, purchase amount and date. You sign the receipt using your personal password and click 'Submit' to proceed with the purchase. Without your password the purchase can't be completed.<br /> <br />This is a system that's been put in place by Visa and MasterCard. It's to provide a more secure service, when making purchases online.<br /><br />Unfortunately, this isn't something we can remove from your Egg Card.<br /><br />Thanks for your message.<br /><br />Emily Stirling<br />Internet Customer Services <br /></blockquote><br />Er yes, thanks for nothing Emily. You don't mention the idiotic implementation or the fact that they are passing liability over to their customers. I'm cancelling my credit card and looking for a secure alternative.<br /><br />Update (2008-11-24) — I can't believe it, the fuckers cancelled my credit card.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com1tag:blogger.com,1999:blog-2590786243514585895.post-28239129921455582522008-11-19T17:44:00.002+00:002008-11-19T17:46:16.185+00:00LWN.net has an interview with us about MinGW Windows cross-compilerHere is the article link if you are an LWN subscriber:<br /><br /><a href="http://lwn.net/Articles/307732/">http://lwn.net/Articles/307732/</a><br /><br />If you're not an LWN subscriber, you can use this free link to get to the article:<br /><br /><a href="http://lwn.net/SubscriberLink/307732/0efc7b75c5696ae5/">http://lwn.net/SubscriberLink/307732/0efc7b75c5696ae5/</a><br /><br />Please consider subscribing to LWN!Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com1tag:blogger.com,1999:blog-2590786243514585895.post-89009252142822838122008-11-09T11:03:00.004+00:002008-11-09T18:09:27.083+00:00OCaml Users Meeting, Feb 2009, GrenobleSylvain is already <a href="http://cocan.org/events/europe/ocamlmeetinggrenoble2009">organizing the next OCaml Users Meeting 4th Feb 2009</a> in <a href="http://en.wikipedia.org/wiki/Grenoble">Grenoble, France</a>.<br /><br />The last meeting (rubbish photo I took below) was a great success, and since so much has happened in the community this year, I expect this one will be even bigger and better.<br /><br /><img src="http://www.annexia.org/tmp/ocamlforum.jpg"><br /><br />Update: <a href="http://le-gall.net/sylvain+violaine/blog/index.php?2008/11/09/47-ocaml-meeting-2008-participants">Sylvain's announcement and the official photo</a>Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-48897335676491050762008-11-02T08:10:00.007+00:002008-11-02T17:38:05.617+00:00malloc failuresI can't put a comment on <a href="http://debarshiray.multiply.com/journal/item/171/RPMlint">Debarshi's post</a>, so I'll answer here. Debarshi complains about <a href="https://bugzilla.redhat.com/show_bug.cgi?id=450011#c8">this comment</a> by the "inimitable" Jeff Johnson:<blockquote><br />You have to look at the usage case, malloc returning NULL is a "can't happen" condition where an exit call is arguably justified.<br /><br />Returning an error from library to application when malloc returns NULL assumes:<br /><br />1) error return paths exist [...]<br />2) applications are prepared to do something meaningful with the error<br /></blockquote><br />Another problem is that only about 1 in 10 memory allocations in a typical C program are mallocs. The rest are stack-allocated variables, and those aren't usually checked at all. If any of your 9 out of 10 stack allocations fail, your whole program fails hard.<br /><br />This is the correct way to deal with those 1 in 10 memory allocations that you can check — provide a custom abort function that the main program can override in the very rare case that they can do anything useful other than exit:<pre><br />void (*custom_abort) () = abort;<br /><br />void<br />lib_set_custom_abort (void (*new_abort) ())<br />{<br /> custom_abort = new_abort;<br />}<br /><br />void *<br />lib_malloc (int n)<br />{<br /> void *data = malloc (n);<br /> if (data == NULL) custom_abort ();<br /> return data;<br />}</pre>Note that the main program can use <a href="http://linux.die.net/man/3/longjmp">longjmp</a> (or exceptions in some cases) to "return" back to a safe point in the program, such as a transaction checkpoint. If the main program uses pool allocators — about the only safe and sensible way to deal with C's programming model — then the program has a chance of recovering.<br /><br />Really the answer is to use a sensible programming language though. Programming languages invented before C had safer, faster memory allocation, dealt with 10 out of 10 memory allocation errors, and provided a mechanism to recover correctly. Those languages are now 30 years more advanced. In 2008 we're having these silly arguments about how to deal with malloc failures. That's a failure of ourselves as programmers.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-42611691078457261562008-10-28T12:34:00.005+00:002008-10-28T13:41:49.981+00:00MinGW: Inkscape cross-compiled from Fedora to Windows<a href="http://inkscape.org/">Inkscape</a>, <a href="http://camltastic.blogspot.com/search/label/mingw">cross-compiled for Windows from Fedora</a>. We use <a href="http://fedoraproject.org/wiki/MinGW">Fedora and a completely open source tool chain for build</a>, and Wine for testing. Windows is only required for final checks before deployment.<br /><br /><img src="http://annexia.org/tmp/inkscape-composite.png">Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0tag:blogger.com,1999:blog-2590786243514585895.post-24554434839896582392008-10-16T16:19:00.003+01:002008-10-16T16:25:03.670+01:00MinGW: It's got an icon and it works!As you can see from the screenshot below, I addressed <a href="http://camltastic.blogspot.com/2008/10/mingw-screenshots.html">Nicu's complaint</a> and added a simple icon to the virsh (virt shell) EXE file. <a href="https://www.redhat.com/archives/libvir-list/2008-October/thread.html#00331">Here's how to do that</a> again using all open source tools. We also a fixed a rather embarrassing endianness bug in our XDR implementation, and so virsh/libvirt can talk to <a href="http://libvirt.org/remote.html">remote libvirtd servers</a>.<br /><img src="http://www.annexia.org/tmp/Screenshot-winxp - Virt Viewer-8.png">Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com2tag:blogger.com,1999:blog-2590786243514585895.post-33614703879782894252008-10-14T14:06:00.004+01:002008-10-14T19:21:59.044+01:00MinGW: Screenshots<a href="http://camltastic.blogspot.com/2008/10/mingw-compile-software-for-windows.html">Previously</a> I showed you how to build software on Fedora which will run on Windows. You can run this software on Fedora using Wine, but it's also nice to know that it even runs on a <i>real</i> Windows machine. Here are some screenshots of the installer running under Windows XP. Remember that this was entirely created on a Fedora Linux system, using completely open source software:<br /><img src="http://www.annexia.org/tmp/Screenshot-winxp - Virt Viewer.png"><br><br />The menubar across the top of the screen comes from virt-viewer. Windows is running virtualized.<br /><img src="http://www.annexia.org/tmp/Screenshot-winxp - Virt Viewer-1.png"><br><br /><img src="http://www.annexia.org/tmp/Screenshot-winxp - Virt Viewer-2.png"><br><br />Notice the desktop shortcuts for each application, added by nsiswrapper automatically. In future we'll actually want to disable some of these since they don't really make sense for command line applications.<br /><img src="http://www.annexia.org/tmp/Screenshot-winxp - Virt Viewer-3.png"><br><br />Who's the "surfer dude"?<br /><img src="http://www.annexia.org/tmp/Screenshot-winxp - Virt Viewer-4.png">Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com2tag:blogger.com,1999:blog-2590786243514585895.post-17330462962784006712008-10-12T20:33:00.009+01:002008-10-12T21:51:11.109+01:00IDN use and abuse<a href="http://jwz.livejournal.com/948180.html">JWZ blogged</a> about the <a href="http://☃.net/">Unicode snowman</a>. If you're running a proper browser, take a close look at the domain name:<br /><a href="http://☃.net/"><big><big><big>☃</big></big></big>.net</a><br /><br />A brief, two sentence overview: For any domain name which begins with <code>xn--</code> followed by some gobbledygook, certain clients like web browsers can interpret the gobbledygook as a <a href="http://en.wikipedia.org/wiki/Punycode">Punycode</a> representation of some Unicode string. So the snowman's real domain name is <code>xn--n3h.net</code>.<br /><br />I was quite excited for a while since many of these Unicode <a href="http://www.fileformat.info/info/unicode/block/dingbats/utf8test.htm">dingbats</a> and <a href="http://www.fileformat.info/info/unicode/block/miscellaneous_symbols/utf8test.htm">symbols</a> are unregistered in combinations of two or more, but then I found that the killjoys at the IETF had put a stop to that with <a href="http://www.ietf.org/rfc/rfc4690.txt">RFC 4690</a>. So while the snowman registration can be continued, no new dingbats can be registered.<br /><br />Nevertheless, we can still have fun abusing the simpler <a href="http://www.fileformat.info/info/unicode/block/cjk_unified_ideographs/utf8test.htm">Chinese characters</a>. For a laugh I registered <a href="http://丄.com/">丄.com</a> and <a href="http://丿乀.com/">丿乀.com</a>. These might not be active when you read this, and to be honest I'm not quite sure where I'll point them at the moment. The first looks like <q>bottom</q>, the symbol for non-terminating programs. Hmmm maybe that'd be good for some insightful blog about functional programming? The second is a total abuse of two characters together, but looks like the <a href="http://ja.wikipedia.org/wiki/8">number 8 in Japanese</a> (IETF rules forbid registering actual numbers, even non-Arabic ones).<br /><br />Someone should do the world a favour and register 丅丨丅.com (<code>xn--9gqa8h.com</code>).<br /><h4>Update</h4>Subdomains are of course not regulated by the IETF jobsworths. Here's another, prettier unicode snowman: <a href="http://☃.earthlingsoft.net/">http://☃.earthlingsoft.net/</a>, and I can have <a href="http://☆☆☆.annexia.org/">http://☆☆☆.annexia.org/</a>Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com1tag:blogger.com,1999:blog-2590786243514585895.post-43162619909591320062008-10-10T10:53:00.012+01:002008-10-10T12:42:35.583+01:00MinGW: Compile software for Windows without leaving your Fedora machineFor the last few weeks I've been focused on the <a href="http://fedoraproject.org/wiki/SIGs/MinGW">Fedora MinGW project</a>. This project gives Fedora users a compelling new feature: you can build your software for Windows, without ever needing to leave the Fedora / Linux environment. In fact you can do everything, up to and including creating a Windows installer for your customers, without needing once to touch Windows.<br /><br />To demonstrate how this works, I'm going to show you how to port a simple application to Windows, using Fedora MinGW. The app I've chosen is <a href="http://freshmeat.net/projects/virtviewer/">virt-viewer</a>, a graphical console viewer for virtual machines, written in C.<br /><br />First we install the cross-compiler environment and any libraries that our program requires. (Until the MinGW packages are accepted into Fedora, you'll have to get them from <a href="http://www.annexia.org/tmp/mingw/fedora-9/">our temporary yum repository</a>)<pre><br />yum install mingw32-gcc mingw32-binutils \<br /> mingw32-gtk2 mingw32-gtk-vnc mingw32-libvirt mingw32-libxml2 \<br /> mingw32-nsis mingw32-nsiswrapper</pre><br />With software such as virt-viewer that is based on the standard <a href="http://www.gnu.org/software/autoconf/">autoconf "configure" script</a>, the cross-compiling step is simple. You just have to do:<pre><br />./configure --host=i686-pc-mingw32</pre><br />That's all you have to do to configure virt-viewer (and most other software) to cross-compile for Windows.<br /><br />Now we just do <code>make</code> and discover ... ah, that it doesn't compile. This leads us to the hard part of porting software over to Windows. Windows uses the Win32 API instead of the usual POSIX / libc API found on Linux.<br /><br />For virt-viewer there are several problems:<ol><br /><li> virt-viewer uses some header files like <sys/socket.h> which aren't found under Win32.<br /><li> We need to include <windows.h> on Windows (but not on Linux). For Win32, this header file is analogous to <stdlib.h> or <unistd.h>, and almost every C source file should include it.<br /><li> virt-viewer makes some Linux-specific system calls which aren't available in the Win32 API. The problematic calls are:<ul><br /><li> <a href="http://www.opengroup.org/onlinepubs/007908799/xsh/usleep.html">usleep</a> (sleep for a specified number of microseconds)<br /><li> <a href="http://en.wikipedia.org/wiki/Fork_(Unix)">fork</a> (create a subprocess)<br /><li> <code>socketpair</code> (create a pipe to communicate with the subprocess)<br /></ul></ol><br />Problems (1) and (2), the missing header files, are easily solved in a very portable way. For each header file which is missing on Windows or Linux, we will just add a configure-time test and some #ifdef magic. Into configure.ac we put:<pre><br />AC_CHECK_HEADERS([sys/socket.h sys/un.h windows.h])</pre><br />and then into the C sources files we put:<pre><br />#ifdef HAVE_SYS_SOCKET_H<br />#include <sys/socket.h><br />#endif</pre><br />and so on.<br /><br />Problem (3) -- missing APIs -- are the hardest problems to solve. In general there are three strategies we could try:<br /><br />(a) Try to find an equivalent but different API which is present on Linux and Windows. As an example here, Windows has a call which is very similar to <a href="http://en.wikipedia.org/wiki/Anonymous_pipe">pipe</a>, and might be used to replace socketpair.<br />(b) Write a replacement function for each problematic API.<br />(c) Comment out the particular feature in the code which uses the missing calls. This is less satisfactory of course: Windows users will now be missing some feature.<br /><br />We're going to fix problems in (3) with a mixture of strategies (b) and (c).<br /><br />Windows doesn't have usleep, but looking at MSDN I see that it does have a function <a href="http://msdn.microsoft.com/en-us/library/ms686298(VS.85).aspx">Sleep (DWORD milliseconds)</a> which can be used as a replacement for usleep.<br /><br />You can test and replace functions conditionally by adding this to configure.in:<pre><br />AC_REPLACE_FUNCS([usleep])</pre><br />Remember that you <i>don't</i> want to replace this on Linux and any platforms that have usleep, and that is what AC_REPLACE_FUNCS does.<br /><br />The code to implement usleep is now placed into a single function in a file with the same name, <code>usleep.c</code>:<pre><br />#ifdef WIN32<br />int<br />usleep (unsigned int usecs)<br />{<br /> unsigned int msecs = usecs / 1000;<br /> if (msecs < 1)<br /> Sleep (1);<br /> else<br /> Sleep (msecs);<br />}<br />#endif</pre><br /><br />The magic of autoconf will ensure this file will only be linked into the main program when it is needed.<br /><br />As for <code>fork</code> and <code>socketpair</code>, it turns out we are quite lucky. These two calls are only used to implement a specific virt-viewer feature, namely tunneling connections over ssh. If you conclude, as I did, that ssh isn't that common on Windows machines, then you can do as I did and just comment out that feature conditionally when building on Windows.<br /><br />With those changes, we have now completed our port of virt-viewer to Windows (<a href="http://hg.et.redhat.com/virt/applications/virt-viewer--devel?cs=2cce513346ae">full patch</a>). After rerunnning:<pre><br />autoconf<br />./configure --host=i686-pc-mingw32<br />make</pre><br />we are left with <code>virt-viewer.exe</code>, a full Gtk application that runs on Windows.<br /><h4>Creating a Windows installer</h4><br />To package up Windows applications into full-featured installers, that include menu shortcuts, desktop icons and an uninstaller, we wrote a little helper program called <a href="http://hg.et.redhat.com/misc/fedora-mingw--devel/?cmd=manifest;manifest=3fc01c067587ae7865b1ae935faad286185832e7;path=/nsiswrapper/">nsiswrapper</a>. As its name suggests, it is a wrapper around the <a href="http://nsis.sourceforge.net/">NSIS Windows Installer</a>, which we also ported over to run natively under Fedora.<br /><br />You'll need to wrap up not just <code>virt-viewer.exe</code>, but the Gtk-related DLLs and helper modules. With nsiswrapper you would do:<pre><br /> nsiswrapper --run \<br /> --name "Virt-Viewer" \<br /> --outfile "Virt-Viewer-for-Windows.exe" \<br /> --with-gtk \<br /> /usr/i686-pc-mingw32/sys-root/mingw/bin/virt-viewer.exe<br /></pre>Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com9tag:blogger.com,1999:blog-2590786243514585895.post-25510356707488767452008-09-20T09:57:00.007+01:002008-09-20T10:51:48.875+01:00What can OCaml do that you can't do in other programming languages?Depressingly predictable <a href="http://tech.slashdot.org/article.pl?sid=08/09/19/1230237">comments on the Haskell story in Slashdot yesterday</a> such as:<blockquote>"I have *never* seen it being used since. To my mind they both belong in the category 'interesting, but pointless'."</blockquote>and<blockquote>"The point is that there's nothing those languages can do that can't be done, often more easily, with the current crop of popular languages. Elegance cannot beat convenience in the workplace, or in most at any rate."</blockquote>and so on.<br /><br />Here are some useful things you can do in OCaml which you cannot do in the "current crop of popular languages".<br /><br /><h4>1. Change the language to suit your task.</h4>In our case the task was to parse binary formats.<br /><br />If you have a few hours to spare, try writing a C program to parse tcpdump files. Don't forget that the endianness in these files is variable so you'll need lots of <code>if (..) { field1 = htonl (field1); ... }</code>. OK so that's a bit hard. Let's say you want to parse a 6 bit length field 'n' followed by an <i>n+1 bit</i> data field (as a 1-64 bit int). Go and write it in C now.<br /><br />Using our <a href="http://code.google.com/p/bitstring/">bitstring extension</a> to OCaml, parsing binary structures is really effortless:<pre><br />let bits = Bitstring.bitstring_of_file "input.data" in<br />bitmatch bits with<br />| { n : 6;<br /> data : n+1 } -> data</pre><br /><a href="http://code.google.com/p/bitstring/source/browse/trunk/examples/libpcap.ml">This is the complete tcpdump parser</a>, just 113 lines of code.<br /><br />And it's fast too. The resulting code compiles down to machine code and inlines <a href="http://camltastic.blogspot.com/2008/08/tip-calling-c-functions-directly-with.html">direct calls to C functions</a> at every opportunity, so in practice you can parse data at speeds approaching C.<br /><br />Another task we had was to check that hundreds of SQL statements in a modest sized web application actually matched with fields in the database, in other words that they didn't reference non-existent fields or treat a string field as an integer and so forth. Doing this by testing is almost impossible because many SQL statements are only used on rare error paths. We could contemplate doing it manually <i>once</i> but not routinely.<br /><br />So instead we extended the OCaml language to do the checking for us <i>every time we compiled the code</i>. The resulting <a href="http://merjis.com/developers/pgocaml">PG'OCaml project</a> gives you compiled-time checked type-safe access to your database. It's used by mod_caml and ocsigen, or you can just use it in standalone programs. I won't go into detail because <a href="http://www.dse.nl/~dario/projects/pgoctut/">Dario Teixeira wrote a great introduction and tutorial to PG'OCaml</a>.<br /><br />Martin Jambon, author of Micmatch which adds regular expressions directly into the language, has an <a href="http://martin.jambon.free.fr/extend-ocaml-syntax.html">excellent tutorial</a>. Browse the <a href="http://caml.inria.fr/cgi-bin/hump.en.cgi?sort=0&browse=92">list of syntax extensions here</a>.<br /><br /><h4>2. Get the compiler to check for errors in your logic</h4>Why can't your compiler check for errors like when you use a read-only database connection in a SQL INSERT statement, or you are supposed to call library function <code>set_word_size</code> because you try to call <code>get_word</code>?<br /><br />In OCaml, using <a href="http://camltastic.blogspot.com/2008/05/phantom-types.html">phantom types [tutorial]</a> you can do exactly this.<br /><br /><h4>3. Drop down to imperative code when you need speed like C</h4> Scripting languages are expressive but slow ... and getting slower. But there's no need to compromise expressiveness to get real speed. Static typed languages with type inference (which excludes C, C++, Java, C# and most others, but includes SML, OCaml and F#) give you the expressiveness to write compact code but <i>without</i> compromising on speed.<br /><br />OCaml programs are more than <a href="http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=python&lang2=ocaml">10 - 15 <i>times</i> faster and less memory intensive than Python programs</a>.<br /><br /><a href="http://cocan.org/features/speed">More about speed and optimizing OCaml programs</a>. Comparison of a <a href="http://www.ffconsultancy.com/languages/ray_tracer/comparison.html">raytracer written in C++ and OCaml</a>.<br /><br /><h4>4. Large well-established standard libraries</h4> OCaml comes with <a href="http://godi.camlcity.org/godi/toc/toc-3.10.html">hundreds</a> <a href="http://pkg-ocaml-maint.alioth.debian.org/debian-ocaml-status.html">of</a> <a href="http://cocan.org/fedora#Package_status">packages</a> for many tasks now.<br /><br />If that's not enough you can directly call out to <a href="http://merjis.com/developers/perl4caml">Perl</a>, <a href="http://ocamljava.x9c.fr/">Java</a>, Python and <a href="http://www.lexifi.com/csml/">.Net [on Windows]</a> libraries. Or you can call C functions directly.<br /><br />You can <a href="http://merjis.com/developers/xphelloworld">compile the same code on Unix, Mac OS X and Windows</a>.<br /><br />So enough of this "elegance can't beat convenience" stuff please.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com9tag:blogger.com,1999:blog-2590786243514585895.post-47131489361838120622008-09-16T17:09:00.003+01:002008-09-16T17:39:14.535+01:00Tip: Read all lines from a file (the most common OCaml newbie question?)Is this the most common OCaml beginners question? It comes up every few weeks on the OCaml beginners list, and I have <a href="http://tech.groups.yahoo.com/group/ocaml_beginners/message/9680">tried to answer it before</a>.<br /><br />It seems that everyone who learns OCaml comes away with the impression that functional programming is the New Cool Thing, and imperative programming is Bad and Must Be Avoided.<br /><br />I'm going to say it now: programming fashions are stupid and counterproductive. The <b>only</b> things that matter are that your program is short, easy to write, easy to maintain and <b>works correctly</b>. How you achieve this has nothing to do with programming fads.<br /><br />Reading all lines from a file is an imperative problem, and the shortest solution (easy to write, easy to maintain and correct<sup>1</sup>) uses a while loop, in OCaml or any other language:<pre>let lines = ref [] in<br />let chan = open_in filename in<br />try<br /> while true; do<br /> lines := input_line chan :: !lines<br /> done; []<br />with End_of_file -><br /> close_in chan;<br /> List.rev !lines</pre>Actually, no, I'm lying. The <b>best solution</b> is this:<pre>Std.input_list chan</pre>which is supplied by <a href="http://code.google.com/p/ocaml-extlib/">extlib</a>. Don't bother to duplicate functions which are already provided in commonly available libraries.<br /><hr><br /><sup>1</sup>This is only strictly speaking correct if you handle clean-up if <code>input_line</code> throws some read error (exception). In the common case where you just exit the program, leaving the channel open is perfectly acceptable.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com5tag:blogger.com,1999:blog-2590786243514585895.post-84169306543504757812008-09-16T16:55:00.004+01:002008-09-16T17:08:01.747+01:00Home server, part 3, quick updateAs I alluded to on the GLLUG mailing list, I got stuck installing Red Hat Enterprise Linux (RHEL) 5.2 on the Viglen MPC-L because of <a href="http://bugs.centos.org/view.php?id=2552&nbn=6">this bug when installing RHEL or CentOS on a Geode LX</a>. Consequently I trashed the Xubuntu pre-install <i>and</i> managed to make the machine unbootable. The BIOS claims to support all sorts of useful boot methods such as booting from USB flash drives and over the network, but in practice nothing seemed to work for me except booting from the local hard disk.<br /><br />This means I'll have to remove the hard disk (it's a 2.5" laptop-sized HDD) and transplant it into another machine, and to do this I'm waiting for <a href="http://www.maplin.co.uk/module.aspx?moduleno=28724">the correct 2.5" - IDE adaptor cable from Maplin</a>.<br /><br />I think the most common question people have asked me is how fast the Viglen is, with its <a href="http://en.wikipedia.org/wiki/Geode_(processor)">AMD Geode LX</a> central processor. When I had Xubuntu on the machine and while I was failing to install RHEL, I played with it and the conclusions are mixed. Firstly it's fair to say that it is not a fast machine, but fast enough to be useful for (eg) light browsing. But I did observe that some specific things are really slow. For example the timezone / graphical map selection during the RHEL install is really slow, but the rest of the RHEL install was perfectly fine. According to Alan Cox the Geode is quite a strange beast. Some pieces of "hardware" are in reality emulated (certain PCI devices for one), so my theory is that when you hit a piece of code which uses these emulated devices in a certain way, suddenly everything slows down.<br /><br />The true test of this will come when I get RHEL installed and have it running as a server.Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com3tag:blogger.com,1999:blog-2590786243514585895.post-2364205165579713472008-09-12T22:38:00.002+01:002008-09-12T22:54:21.408+01:00Home server, part 2, just photosI received the <a href="http://www.viglen.co.uk/viglen/Products_Services/Product_Range/Product_file.aspx?eCode=XUBUMPCL&Type_Info=Description&Type=Desktops&GUID">Viglen MPC-L</a> which will be part of my <a href="http://camltastic.blogspot.com/search/label/homeserver">home server</a>.<br /><br />The first, rather unfair impression is "cheap Mac Mini", but to Viglen's credit it is very much cheaper than a Mac, about one quarter of the price, at a mere £80 inc. tax and delivery.<br /><br /><img src="http://www.cookingwithrichard.com/wp-content/uploads/2008/09/computer1.jpg"/><br /><br />Front and back:<br /><br /><img src="http://www.cookingwithrichard.com/wp-content/uploads/2008/09/computer2.jpg"/><br /><br /><img src="http://www.cookingwithrichard.com/wp-content/uploads/2008/09/computer3.jpg"/><br /><br />The full set is shown below. I was surprised that Viglen even bothered supplying a mouse and keyboard, since they don't supply the obvious <a href="http://en.wikipedia.org/wiki/Image:Amstrad_CPC_Advert.png">missing bits</a>:<br /><br /><img src="http://www.cookingwithrichard.com/wp-content/uploads/2008/09/computer4.jpg"/><br /><br />... but what about all this waste? Viglen could do better to reduce this:<br /><br /><img src="http://www.cookingwithrichard.com/wp-content/uploads/2008/09/waste.jpg"/>Richard Joneshttp://www.blogger.com/profile/08315526595922432607noreply@blogger.com0