sa-update

SpamAssassinFor quite a while my SpamAssassin installation for Windows won’t update anymore. It simply quit with some exit code 4. Unfortunately, SpamAssassin is running on this server for quite some time and often one simply forgets about systems that are up and running until they fail. Having a closer look at the error message it appeared that sa-update had some problems with some of the preconfigured channels.

C:\SpamAssassin>sa-update.exe -v --nogpg --channelfile UpdateChannels.txt
channel: no 'mirrors.khop-bl.sa.khopesh.com' record found, channel failed
channel: no 'mirrors.khop-general.sa.khopesh.com' record found, channel failed
channel: no 'mirrors.khop-sc-neighbors.sa.khopesh.com' record found, channel failed
Update failed, exiting with code 4

C:\SpamAssassin>if NOT EXIST .\share\3.003001\updates_spamassassin_org.cf EXIT /B 1

A look at the sa-update documentation reveals there might be a problem with downloading and extracting the updates.

An exit code of 4 or higher, indicates that errors occurred while attempting to download and extract updates.

As the sa-update.bat refers to a file called updatechannels.txt you probably find the failing channel entries there.

# see http://khopesh.com/wiki/Anti-spam#sa-update_channels
# for more information

updates.spamassassin.org
khop-bl.sa.khopesh.com
khop-general.sa.khopesh.com
khop-sc-neighbors.sa.khopesh.com

Digging a little bit deeper showed that the channels provided by Adam Katz at http://www.khopesh.com/ became unmaintained. Therefore, the easiest way to get rid of the failures is to remove the entries from the file for good. Eventually, this will leave you with only the official channel at updates.spamassasin.org.

Do not forget about your Crosscable when updating an EZControl XS1

No SEO friendly title in my mind, there was nothing better to describe tonight’s uproar while upgrading some part of my home automation environment.

I don’t always change a running system. But when I do, I break it for sure. And of course, when I change something, I always change literally everything at once.

While following some rules strictly during my work day, such as

  • never change a running system ,
  • change one thing at a time and
  • don’t use beta software in a production environment

I definitely forget about those rules once I am at home. Therefore, I fiddle around with my home WiFi for days, upgrading everything to IEEE 802.11n including a new full duplex repeater and new access points with exotic chipsets requiring very specific build of DD-WRT.

Still debugging slow pings, lags and randomly missing segments of my network, I also decided to upgrade my EZControl XS1 to the very latest available (beta) firmware.

Once I shut down the device in its bootloader mode, and tried to uploaded the latest firmware (via browser), the HTTP request failed and every single browser on any OS told me the page wasn’t able to be displayed. Once back to the bootloader, it told me the uploaded firmware check failed. Hitting the browsers refresh button did not help a lot either. That’s the way how you brick a $300 gadget in seconds like a boss.

Crossover CableAs anybody would do in such a moment, I dug into my box with long forgotten network cables and picked the very only crossover cable probably exiting within the range of two hours driving. Hail to me, still owning one.

No kidding, most of todays kids won’t even know such a thing did exist. Actually, it’s the sort of cable one used long time before wireless. Before the age of switches, routers and hubs at a time when one tried to connect to machines via ethernet directly.

Connected to the XS1 via the crossover cable, there is absolutely no problem uploading the firmware. When performing the same operation via wireless, there goes something wrong in layer 2 or 3 or maybe 4 – or not. We eventually will never know.
Next time you are going to upgrade the firmware of any device via network, keep your crosscable in reach, you are probably going to need it.
Next time, I won’t change a running system, always change one thing at a time and I will never use betas in production environment… I don’t think so.

openHAB EZControl Integration

For quite a time, I monitored the openHAB project. Eventually, I found some time to test it out and tried to set up some of my existing home automation hardware with openHAB.

Therefore, I decided to set up my EZControl XS1 as a very first testbed.

Learning it the Hard Way

Before digging into the details, I’d like to share some of my experience during my very first hours with openHAB.

First of all, the contributors (currently counting 38) have done a great job in documenting the project. Installation and configuration does work quite well. The same for the basic setup of the development environment (e.g. in the case you want to implement an own binding).

Unfortunately, there are many details you need to know to get the system up and running, not being offered at a glance. You have to dig into groups, the wiki, demo files and perform a lot of trying, experience a lot of frustrating errors. Once you figured out how it works, it is not hard to remember. Done once, some of the issues are that simple, that it is clear why they are not mentioned in the wiki at all. Unfortunately, if you haven’t dealt with openHAB before, you have to learn it the hard way.

Not to forget, here’s a list of sources you definitely need to check in case you stuck with your very first setup:

Definitely, do not miss the last one. Many of your questions will be answered in the examples wiki, though. Still, some details might be missing, but that’s the place to check first.

Said that, there’s no critics about the documentation. It’s great, you just see it’s written by those who implemented it. And as such, one sometimes forgets about the trivial bits and bytes one need to know to start.

Setting up the EZControl XS1

Most devices I run on my EZControl are sensors from the ELV FHZ 1×00 System and intertechno wireless receivers.

For my convenience, I uploaded the configuration app to my internal web server though.

XS1 Setup

For now, I am only interested in the sensors available at the sensors tab.

Sensors

Iwanted to be independent from the data structure to set up the XS1 with openHAB. Therefore, the only thing you need to know, is how you named the sensors, though.

Switches

Before starting, make sure the HTTP binding is installed as it is used both, for switches as well as for sensors.

To address switches, you can use the REST interface of the XS and address the switches directly, you simply need to know the number of the switch being used within the query parameters such as number=2 within your *.items configuration file..

Switch Switch_Balcony_gBalcony "Balkon" (gBalcony) {http=">[ON:GET:http://192.168.0.242/control?cmd=set_state_actuator&number=2&function=1] >[OFF:GET:http://192.168.0.242/control?cmd=set_state_actuator&number=2&function=2]" }

Sensors

Sensors can be fetched from the ES1 using HTTP binding as well. Therefore, I pick up the XML output from the ESZ, in my case provided at http://192.168.0.242/xml.

XS1 XML

To pick a corresponding value of a sensor, I’ve created a XSL transformation

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:output indent="yes" method="xml" encoding="UTF-8" omit-xml-declaration="yes"/>
    <xsl:template match="/">
        <xsl:value-of select="xml/data/*[text()='Arbeitszimmer_Temp']/following-sibling::node()"/>
    </xsl:template>
</xsl:stylesheet>

The XSL is quite simple, picking up he value from the next node after the node with the given sensor name. That way, structural changes in the XS1 setup or even a replacement of a sensor do not affect the openHAB configuration.

Create a XSL for each sensor value, you plan to use within openHAB and place them into the transform folder under configurations in the openHAB folder structure.

XSLT Transformations

Again in the *.items configuration file, using the HTTP binding, I pick up the XML for each node and apply the transformation provided, resulting in a single value.

Number Temp_gOffice "Temperatur [%.1f °C]" (gOffice) { http="%lt;[http://192.168.0.242/xml:10000:XSLT(Arbeitszimmer_Temp.xsl)]"}
Number Hum_gOffice "Feuchtigkeit [%.1f %%]" (gOffice) { http="<[http://192.168.0.242/xml:10000:XSLT(Arbeitszimmer_Hum.xsl)]"}

While this works quite well, the XML is fetched from the XS1 for each value node. Looking forward for version 1.3 of openHAB, there is planned to provide a cache, so the XML would be fetched only once while the binding would fetch the data from the cache.

Sensor Values using the HTTP Binding

Done once, adding new sensors or switched using the XS1 takes just a few minutes. Changing the .*items configuration by adding new HTP bindings, copying and modifying the XSLT and adding of course adding the sensors and switches to the XS1.

SOAP with Python

To evaluate a SOAP service, I decided to increase my Python skills by one by consuming the SOAP service with Python. SOAP Web Services from Dive Into Python seemed a quite good starting point with detailed instructions for setting up your Python environment to do so.

After starting to work thought this article how to install the required SOAP libraries, I realized quickly links are broken and various information became obsolete.  Therefore, I decided to summarize the amendments to set up SOAP libraries for Python, in this article focusing on how to set up PyXML, fpconst and SOAPpy.

Prerequisites
Before starting lets be sure Python is installed. You might check with

where python

to determine which python will be accessed via the PATH variable.In my case it is a Python 2.7 running on Windows 8.0.

Python 2.7 on Windows 8.0PyXML
PyXML can’t be accessed via http://pyxml.sourceforge.net/. Instead you have to access the site via http://pyxml.sourceforge.net/topics/ directly. Unfortunately, you might realize that the latest supported version from there is Python 2.4. You will find a Windows Installer for PyXML 0.8.4 for Python 2.7 at the OddThinking blog instead.

Run through the installer, just make sure, you pick the right Python installation during the process.

PyXML Installation DialogVerifying the installation is quite simple using the Python console, though.

>>> import xml
>>> xml.__version__
'0.8.4'
>>

fpconst
Instead of picking up the fpconst package from Unversity of Washington’s broken link,  you’ll meanwhile find fpconst 0.7.2 within the Python Package Index. Download and extract the archive to any temporary folder. Change folders to the extracted one and run

python setup.py install

from the console.

fpconst Package installationAgain very the installed package with

>>> import fpconst
>>> fpconst.__version__
'0.7.2'
>>>

SOAPpy
SOAPpy is probably the package causing the most headache. I first picked SOAPpy 0.12.0_rc1 from the download section at  http://pywebsvcs.sourceforge.net/. It caused more or less trouble, trying to install the package. In addition, SOAPpy was announced to be merged into the ZSI (Zolera SOAP Infrastructure) web service toolkit. If you are going to try ZSI anyway, download and installation are similar to the example above using

python setup.py install

Even the latest version of ZSI installs fine with Python 2.7.

Anyway, when sticking with SOAPpy, you might want to get the latest version (0.12.5) from the Python Package Index. Extracting the archive and following the steps above will result in an import error, though.

Traceback (most recent call last):
  File "setup.py", line 7, in <module>
    from setuptools import setup, find_packages
ImportError: No module named setuptools

To solve this, navigate to the setuptoools 0.9.8 in the Python Package Index, and download ez_setup.py to your local machine. Run the script with

python ez_setup.py

which will compile quite a bunch of stuff on your local machine. Follow the instructions, and add the C:\Python27\Scripts to your PATH variable.

Once installation is complete, you will find an easy_install.exe program in your Python Scripts subdirectory. For simple invocation and best results, add this directory to your PATH environment variable, if it is not already present.

Head back to the download folder of SOAPpy 0.12.5 and run once again

python setup.py. install

which should finally end up in something similar to

...

Installed c:\python27\lib\site-packages\docutils-0.11-py2.7.egg
Searching for fpconst==0.7.2
Best match: fpconst 0.7.2
Adding fpconst 0.7.2 to easy-install.pth file

Using c:\python27\lib\site-packages
Finished processing dependencies for SOAPpy==0.12.5

For the very last time, verify the installation by

>>> import SOAPpy
>>> SOAPpy.__version__
'0.12.5'
>>>

By now, you should be ready to use all three packages with your Python 2.7 installation.

Coll Stuff #2

Some more stuff I came along the last few days…

Feedback Loops in Scrum the Other Way Around

Scrum in ActionUser Stories are the way a non-technical product owner can communicate with the technical team. It is probably the best way any product owner can tell the technicians what he or she actually wants. A the same time it is the best point to pick up technical requirements, limitations and pitfalls related to your requirements. Therefore, take this opportunity and listen carefully. The feedback loop is not only top down, it’s bottom up as well, you just have to use it.

A difference maybe not realized yet

The difference in Scrum related to many other development processes is the rapid feedback you might get during backlog grooming or estimation meetings. Developers will moan about the stories, complain about requirements and tell you more often than not, these particular wishes might be very expensive or even not possible to achieve at all…

Take Action

Even this might be exaggerated, listen because here you get feedback from the folks knowing the software best. Change requirements on the stories, modify the scope, prioritize impossible features down. Nobody else does understand the overall system that well. Non of your architects, managers or analysts understand the system that way.

No means Maybe Later

No does not mean no by definition. If you come up with a story, developers think its impossible, it’s actually not. Stop fighting for the story. Take it, go back to the drawing board and refine or start over. No simply means it too expensive. Always remember: We are talking about technology here. probably everything is possible. It is simply a questions about how much effort one (in this case the product owner) is willing to put into it. If developers say no, they come up with a natural instinct which tells them it simply not feasible under the current circumstances. Always remember, SCRUM is about performing on the very best level under a certain set of circumstances and limitations at a very specific point in time.

Resume

As a product owner make use of estimation meetings and/or backlog groomings. Go over stories again an again and pick up the feedback from your developers. With every meeting you will get better feedback more details and better estimations. Take this opportunity receiving rapid feedback from the developers. There is easier way to get this feedback, though.

Cool Stuff

Every week, there are many things I stumble upon worth blogging about. Unfortunately, there is either too much to at that time, just too little to say about it or it’s simply worth to be remembered without bloating it with an exhaustive blog entry.

Therefore, I want to give a more or less regular (let’s see how this works out) update on all these cool stuff one might find out there in the wilderness of the cyberspace.

 

Where is my Space

Using Bootcamp, running a Mac OS X and a Windows installation in parallel, space is slowly but constantly running low on my MacBook Pro. Before upgrading my HDD, I was looking for a better way than using du or df -h to check where I might free up some space on my disk.

Disk Inventory X is a free tool available using cushion tree maps to visualise the disk usage. A similar tool was developed at the Technical University of Eindhoven named SequoiaView. While SequiaView was developed for Windows, Disk Inventory X is designed for Mac OS, working fine with version 10.8.3.

Disk Invetory X

After selecting the disk, it’s possible to browse through the directory structure analysing where your storage when to.

Disk Invetory X Cushion Tree

Both, SequoiaView for Windows and Disk Inventory X for Mac are quite useful programs that should be definitely part of ones toolset.