mobile phone: how much of a SIM servant?

May 21, 2012

It does not seem so, but the SIM (UICC in general) is quite powerful. It’s not just a phonebook and subscriber information storage, but it’s more like a micro-controller. It can run small applications (cardlets) and use the phone as communication device.

Using CAT (Card Application Toolkit, the more general approach for USAT, SAT, and STK) cardlets are able to display information on the screen, send SMSs, and establish Internet sessions. But they can also divert calls, secretly add participants to the conversation, or even ask for GPS coordinates. That is, if the phone supports such requests. These capabilities are sent by the phone to the UICC in the Terminal Profile. The APDU is defined in ETSI TS 102 221 §11.2.1, and the meaning of it in ETSI TS 102 223 §5.2. While it is well defined, it is very unknown and more importantly, there is no easy way to know what a phone is capable of.

But SIMtrace allows you to listen to the communication between the phone and the UICC and get the terminal profile. We can now collect these information and create a database (under CC-BY-SA v3.0). To ease the submission, a script will parse the output of the simtrace application, detect the terminal profile, and ask for some more details about the phone before submitting it. You can also decode the submitted terminal profile and find out what features are supported by the phone, and which classes it is compliant to.

More information is available at the project site (or, and everyone (owner of a SIMtrace or not) is invited to contribute to it.

additional: SIM picture, phone picture.


ebook owner

May 17, 2012

I recently bought an ebook from immatériel; a great ebook store because it has an wide range of electronic books, is not amazon, and they offer DRM free epubs.

When I opened it, I saw a “Property of Kévin Redon (my@email.address)” on the first page. First I was surprised they personalize the book, and I said myself it’s as we do in paper books, we write our name at the beginning so when we lend/give it, the next reader knows where it came from.

But then I saw it at the ending of each chapter. So they don’t do it to please you, but to mark the book (instead of using DRMs) so you don’t spread it in the wild anonymously. While it was useful to have the name in the paper book, so it would be brought back to the original owner, with electronic book this is not necessary. It’s not physical, when you share it, you create a copy, and still have yours.

Having my name at the end of every chapter was getting annoying, so I decided to remove it. As epub is basically a zip archive of html pages, it was quite easy and fast. I also discovered they put your customer ID as HTML comment along with the name, and also got rid of it. So here is the script doing that (but only for immatériel ebooks).


A-GPS data for TripMate 850

April 20, 2012

The GPS device Transystem TripMate 850 is a nice GPS logger. It has:

  • a MediaTek MT3329 GPS chip (with very good reception)
  • logs onto a miroSD card (~unlimited logs)
  • has a LCD screen (great to check the speed in planes)

But is has also some drawbacks:

  • it uses 2xAA batteries and is not USB rechargeable
  • the USB port is just to read the microSD content (very slowly), and does not provide GPS mouse capabilities
  • bluetooth cannot be switched off (drains quite some battery)

It also comes with a software which downloads AGPS data (which you can then put on the microSD card for the GPS).

Because it’s a Windows software (which also works fine under wine), I though I could have a look at how it gets this AGPS data. And it only took a single wireshark session to find out where the data it from. And here is the link:


Now you can simply use wget to update the AGPS data.


SFR statistics

March 3, 2012

Update (2012-04-22)

Here some updates:

  • The feature is still available, and I got no reply. They even reworked the interface, so to be more square
  • Every night the web interface is down (for technical reasons), but the ajax call still works (I think it’s more for practical reasons)
  • The interface will only tell you the phone model when it’s compatible with the femtocell, but the underlying json response will always tell the model (if it’s an SFR subscriber)

  • Previously the scraping script just saved the plain json response into a file, and the extraction script collected them. But beginning with 500 000 files, the extraction took hours because of the high disk IO. Now the script parses the response and directly saves it into a SQLite3 database
  • It seems you don’t even need the femtocell option (even if it’s for free). You only need an SFR account, and just have to log in. The new script can do the log in for you, if you enter the credentials. Else you must provide a valid cookie
  • Since a SQL database is used now, making statistics has become a lot easier. The new statistics will do the pie chart for you (using the pie script). Both are included, and you can reuse the pie script to make nice (vector) charts for whatever you want

Here the scripts/source code: sfr_phones_db.tar

And just for the fun: new statistics (based on 3.88% of all numbers, collected over the last 2 month).

Original (2012-03-03)


It all began when I logged on the SFR customer portal, and there my phone was shown. This phone was my own, and not the one provided with the mobile subscription. How do they know which phone I use?
Well, every time your phone registers to a cell tower, it can send its IMEI (deliberately, or upon request). This IMEI reveals the manufacturer and model of the phone (the prefixes are registered), and uniquely identifies it.
And so SFR knows which phone their subscribers are using.
This is not something new, but it was the first time I saw an operator show this info to the customer.

Now, I also have a femtocell from this provider (you can request it for free), and I am allowed to add subscribers to the group of phones allowed to connect to the femtocell. SFR has to verify if it’s a SFR customer, and additionally tells if it’s compatible with the femtocell. Only 3G can connect to the device as it provides UMTS coverage. And this information is shown when trying to add a phone number. Even the phone model of the subscriber was shown (quite a privacy issue).


Thus, I though I could put in any number, so to know if it’s an SFR number, and if the subscriber uses a 3G phone, an which one. But I don’t want to do this manually using their complicated website, when a script could do the job for me.

To find out how this feature works, first go to the specific section:

  • path: Mon Espace Client > Gérer mes lignes > Ma ligne mobile > Offre et Options > Choisir de nouvelles options
  • click: ESPACE CLIENT -> Gérer mes lignes -> Mon abonnement et mes options -> SFR FEMTO; Créer Mon groupe SFR Femto -> Modifier mes membres -> Ajouter un numéro mobile SFR
  • URL:

Then simply start wireshark and sniff the HTTP traffic (they do not use HTTPS). There you can see that once you entered a phone number in the form, some AJAX script performs a request.
It very simply queries the following URL, with MSISDN being the phone number.

The server returns a json response with the relevant information: is it an SFR subscriber, is it femtocell compatible, and what is the phone model.


I decided to use (abuse) this capability in order to make some statistics (with self-made pie charts):

  • how many phone numbers are registered to SFR
  • how many phone are 3G phones (implied from being femtocell compatible)
  • what are the most used phone manufacturer and model

The (ruby) script uses wget to save the json responses, and only requires a valid cookie. The get this cookie, just log on the SFR customer portal and go to the page to add numbers to the femtocell group. You can export the cookie using Export Cookies (for Firefox).

In the beginning, the script randomly queried numbers in the range 06xxxxxxxx (100 millions possible numbers). It got 0.32% of the possible numbers between 2011-09-29 and 2011-10-05.

Here the results:


On 2012-03-02, 0.05% of possible numbers have been scanned again (randomly). The results are quite similar.


Later on, I found out phone number ranges where pre-allocated. In France, the ACERP is responsible for that (regulating telecommunications), and the list is available on their page.
Now instead of trying random numbers, it’s possible to use the known ranges. SFR has 35 prefixes in the 06xxxxxxxx range, with in total 27.8 millions possible numbers (out of 100 millions).
But there is a glitch. Since a few years, the subscribers are allowed to keep their number when switching operator. This is the so called “mobile number portability”. Thus some of the numbers with SFR prefixes are not SFR subscribers anymore, and SFR has subscribers with non SFR prefixes. This is not regulated by the ACERP and there is no public list. Only the operators themselves know which operator uses one of their numbers. 14.80% of SFR numbers do not have an SFR prefix (from the statistics from 2012-03-02), which is not really insignificant.

Out of the 27.8 millions numbers with SFR prefixes, 0.46% have been queried on 2012-03-03.


Obviously I can not provides the data collected, because of privacy reasons. Also, only the statistics have been kept (not all shown here), an the raw data deleted (I’m quite responsible, aren’t I?). But here are the scripts.

Finally, I don’t know how often the subscribers phone information are refreshed. One could find it out by trying, but I’m too lazy.


Now the boring legal thoughts.

I contacted SFR, but that was really not an easy task. On the official contact page there is no e-mail address. You can only phone the customer service. It’s quite expensive, reserved for customers, French only, and I really don’t want to talk to a hot-line trying to explain I don’t have a problem with my product, but they have a privacy issue. There also is a forum, but you have to subscribe and the post will be public (we want to warn them only). Also, I’m not a friend of dead trees, don’t want to pay a 1€ stamp for a letter, and why should I provide them my postal address.
After some time (1 hour of clicking), I randomly found (using google, not their site) an e-mail address on the impress of the corporation site: contact[at] Sadly it is invalid (their mail server returns a unknown recipient error), as are root/admin/webmaster/postmaster.
Finally the mail successfully went to info[at], but even after one week I got absolutely no reply and the feature is still available.

On one hand, the corporations always tell us to warn them if we find a bug. But when we try to do so, it’s really hard to get in touch, and most of the time, as long as the bug is not public and does not cost them too much money, they simply do not care. This is not the first time this happens to me. Actually, I never got any reply until the bugs I reported got in the mainstream specialized press.

Now another question: is this work legal? This is a hard one. On one hand, scraping/harvesting is not really forbidden. All search engine do it (as long as they respect robot.txt). It’s on their website, which is available over the whole Internet.
On the other hand, you have to login in. This function is only available for customers (a friend). But then, it’s a service provided by the operator himself. They implemented and thought about it. It’s a feature, not a bug. The operator is responsible for its data, and making user data available to others is probably breaking some privacy laws.
Now about copyright. I extracted data from their website. Considering a website as a database is risky, but this feature is somehow connected to a real database. In the copyright laws, there is a section about databases, with different right. Extracting a database is less allowed. But the script is not really connected to it, merely to a simple interface, available as service on their website. Also, I did not keep or published the data. I just ran some statistics over it and erased them afterwards.
Finally, it’s a French website, but I’m not doing this from France (luckily I don’t live there). Which law applies to me? The French or the local one?

So coming back to the question: it’s a gray zone open for debates. But getting this statistics was a nice exercise with interesting results. How else would you know how much of the allocated numbers are used? how many have 3G phones? what are the main brand? … This is data normally only operators have and would never make available. I’m not even sure if they do such statistics or if it’s just data laying around. And I think I was quite responsible when handling it.


television on Ubuntu

April 29, 2011

This post is an update of Television on linux.
anttip stopped the support (repository) of the ec168 dvb-t chip as it got integrated in the kernel.

But Ubuntu 11.04 does not come with it, so we have to compile the module and install it :

# install firmware
sudo wget -O /lib/firmware/dvb-usb-ec168.fw
# dependencies and linoux kernel source
sudo apt-get install linux-headers-`uname -r` linux-source
cp /usr/src/linux-source-`uname -r | sed "s/-.*//"`.tar.bz2 .
tar xf linux-source-`uname -r | sed "s/-.*//"`.tar.bz2
# configure the kernel
cd linux-source-`uname -r | sed "s/-.*//"`
cp /boot/config-`uname -r` ./.config
sudo cp /boot/`uname -r`
cp /usr/src/linux-headers-`uname -r`/Module.symvers Module.symvers
sed -i 's/# CONFIG_DVB_USB_EC168 is not set/CONFIG_DVB_USB_EC168=m/g' .config
sed -i '/CONFIG_DVB_USB_EC168=m/ a\
CONFIG_DVB_EC100=m' .config
make oldconfig
EXTRAVERSION=`uname -r | sed "s/[0-9\.]*\(-.*\)/\1/"`
# make the modules
make kernel
make modules M=drivers/media/common/tuners
make modules M=drivers/media/dvb/frontends
make modules M=drivers/media/dvb/dvb-usb
sudo make modules_install M=drivers/media/common/tuners
sudo make modules_install M=drivers/media/dvb/frontends
sudo make modules_install M=drivers/media/dvb/dvb-usb
# install the modules
sudo modprobe dvb_core
sudo modprobe mxl5005s
sudo modprobe ec100
sudo modprobe dvb_usb_ec168

The rest is the same. I still use kaffeine to use it.


mp3 resume trick

March 5, 2011

Philips SA3125/02

My iPod shuffle died a few weeks ago. I used it to listen to podcasts. Now I need a new one. I thought of buying one from DX, but that might not be a good idea.
I doubt the quality of these devices. Additionally of being small, lasting long (>10h) and having a mini-USB connector, I need a mp3 player also able to play ogg and flac files, simply because I like open standards and quality. And because podcats are long, it should be able to resume play even if powered off. I don’t need any fancy mp4 player, but at least it should have a one line screen, to know which podcast is currently playing.

I rapidly looked on eBay, and I found the SanDisk Sansa Clip, which has everything I need, and even more (good support).
After having failed 5 times at bidding because I do not wish to pay more then 20€, I just impulsively bought a Philips GoGear SA3125/02. Because the description was really poor, I got it cheaply for 12€. I didn’t know the exact model, but because it comes from a known brand, it should not be so bad.
Well, what a deception. It does nothing I wished for. At least it is small, despite the big screen taking half of the space, and uses UMS over USB. But it does not support ogg or flac, nor is it able to resume. It is slow, does not handle directories and even when browsing while playing, the play restart when going back.

Until I get a decent mp3 player and sell this one, I still can use it. Because it can at least remember which track was played last time, I though of a trick to do some sort of a resuming : cutting the podcast in 5 minutes pieces. So I wrote a script doing that automatically (using ruby, ffmpeg, liblame, and id3v2).

#!/usr/bin/env ruby
# duration in seconds of each split
SPLIT = 5*60
# verify input
raise "specify file input" if ARGV.size==0
file = ARGV[0]
raise "#{file} does not exit" unless File.exist?(file)
# get information
info = `ffmpeg -i #{file} 2>&1`
# get duration string
# example : 02:01:02.89
duration = ""
info.each_line do |line|
  if line =~ /Duration/ then
    duration = line.split("Duration: ")[1]
    duration = duration.split(",")[0]
# transform to duration is seconds
seconds = 0.0
duration = duration.split(":")
duration.each_index do |i|
  seconds += duration[i].to_f*(60**i)
seconds = seconds.ceil
seconds *= 1.0
# get title
title = nil
tags = `id3v2 -l #{file}`
tags.each_line do |line|
  if line=~/^TT2/ then
    title = line.split(": ")[1..-1]*": "
# create dir
dir = file.split("/")[-1].split(".")[0..-2]*"."
Dir.mkdir dir unless File.exist? dir
# split audio file
(seconds/SPLIT).ceil.times do |i|
  id = (i+1).to_s.rjust(3,'0')
  split = dir+"/#{id}_"+dir+".mp3"
  puts split
  `ffmpeg -i #{file} -ab 128000 -ss #{i*SPLIT} -t #{SPLIT} #{split}`
  `id3v2 -T #{i+1}/#{(seconds/SPLIT).ceil+1} -t \"#{title} #{id}/#{((seconds/SPLIT).ceil+1).to_s.rjust(3,'0')}" #{split}`


September 10, 2010

arte is one of the TV channels I like because of some good documentations (not all). Some years ago arte started arte+7. The VOD webservice were you could watch the shows produced by arte up to 7 days after TV broadcast.
At the beginning it offered the option to view the video using flash or windows media player (WMP). Thus it was easy to record the streamed video using :

mplayer -dumpstream mms://...

around 6 month ago, arte+7 stop offering the WMP option, but the mms streaming server was still working.
since 1 month ago (or longer), even the streaming server is down. The only solution is the flash player. But with the “illegal” rtmpdump it’s still possible to record it (here the ppa).
There is also a plugin for totem to watch the shows, but here I will focus on the script to download it.
The original comes from this thread, I just enhanced it a bit. I discovered it after I almost finished my implementation in ruby. This one requires less, is smalled, and works great

# language
if [ -z $LANG ]; then
# show all url
# page url
# save page because use two time (for the player and xml1)
wget -o /dev/null -O $HTML_PAGE $URL_PAGE
# url of the player
URL_PLAYER=`sed -n 's/<param name=\"movie\" value=\"\([^\?]\+\)\?.*/\1/p' $HTML_PAGE`
if [ -z $URL_PLAYER ]; then
	echo "could not find video player url"
	exit 1
# xml1 = description of the video + link to the xml containing the video url
URL_XML1=`sed -n 's/vars_player.videorefFileUrl = \"\([^\"]\+\)\";/\1/p' $HTML_PAGE`
if [ -z $URL_XML1 ]; then
	echo "could not find video description 1 url"
	exit 1
# xml2 = detail description of the video + link to the video
URL_XML2=`wget -o /dev/null -O- $URL_XML1 | sed -n "s/<video lang=\"$LANG\" ref=\"\([^\"]\+\)\"\/>/\1/p"`
if [ -z $URL_XML2 ]; then
	echo "could not find video description 2 url"
	exit 1
# rtmp link for the video
URL_RTMP=`wget -o /dev/null -O- $URL_XML2 | sed -n 's/<url quality=\"hd\">\([^<]\+\)<\/url>/\1/p'`
if [ -z $URL_XML2 ]; then
	echo "could not find rtmp url"
	exit 1
# name of the video
VIDEO_NAME=`wget -o /dev/null -O- $URL_XML2 | sed -n 's/<name>\([^<]\+\)<\/name>/\1/p' | head -n 1 | sed -e 's/\//-/'`
if [ -z $URL_XML2 ]; then
	echo "could not find video name"
	exit 1
# html page not used
if [ "$VERBOSE" = 1 ]; then
	echo "lang : "$LANG
	echo "url : "$URL_PAGE
	echo "player : "$URL_PLAYER
	echo "xml1 : "$URL_XML1
	echo "xml2 : "$URL_XML2
	echo "rtmp : "$URL_RTMP
	echo "video : "$VIDEO_NAME
# record the stream
rtmpdump -r $URL_RTMP --swfVfy $URL_PLAYER -o $VIDEO_NAME.flv


July 18, 2010

I own an iPod. So what ? Everybody does.

Actually I do not like Apple. I found this iPod shuffle (1st generation) lying on the street, and I kept it because the hardware is not bad at all. It is very minimalistic: no display, only the required buttons to play, and integrate battery lasting for long hours, and it also can be used as a USB memory stick.
But the software from Apple is not of my taste. I like to be able to decide for myself what is good for me, and keep my freedom. Thus I do not use iTunes at all.

Normally iPods are designed to work with iTunes only, but there is a way around. On linux, the software for the iPods is GTKPod. But this one does not handle the iPod shuffle because of it’s unusual minimal system. But again, there is a small python script capable of updating the database so the player can play new tracks.
Now I can just use the iPod like a classic MP3 USB memory stick, just drag and drop the files, and don’t forget to update the database.

But after installing Ubuntu 10.04, The UMS (USB Mass Storage) did not show up any more. dmesg showed that the SCSI device is attached, and I can mount it manually. Thus the fact that automount does not work is probably due to udev. And actually it is. The cause is libgpod-common inserting corrupted rules. Removing it resolves this issue.

Now it works fine again, an it’s even easier to use then the “normal” users do.


Internet for everybody : un peu, beaucoup, à la folie

June 10, 2010

My intention was good, I wanted to share my Internet. Completely opening my WLAN was too dangerous (network security, privacy issues, …), so there should be a second WLAN, open and available for external users.
But also, I wished the guest not only to profit from my sharing, but also to do the same and share it’s own Internet, if available.

This is when I found FON. It was selling a router (fonera 2.0(g)) fitting perfectly my needs :

  • it’s a small nice router, with USB
  • offers a lot of features (NAS, network printer, …)
  • based on openwrt (an open source distribution)
  • creates a private WLAN
  • offers a public WLAN
  • other foneros (fon members) have free access to the Internet I share
  • used all over the world

Nice isn’t it ? And this is why I participated to this system for around 2 years, without thinking too much about anymore.

After a certain time, and when my eyes rediscovered the router lying there, I realized that I never used another fonspot. Not that I never tried, I’m traveling often and I always struggle finding a free access point. This is when I re-thought about the FON offer and found more and more negative points (ascending in importance).

  • their wiki was closed (every contribution become their property)
  • the router software became more and more a stupid Internet-consumer tool : plugins for facebook, twitter, megaupload, picasa, flickr, … (I hate centralized services)
  • the FON had some issues with the open source community : they had sign images. now they are still in use, but not enforced
  • the lying DNS which redirected me when Internet connecting crashed was really annoying
  • and now the most important : for non-fonero, the internet access was only possible when paying FON (buying hour based tickets). Why would FON earn money over MY Internet connection.

Because I had some spare time (actually I’ve made some), I wanted to play with OpenWrt and do my own installation. This is also when I discovered freifunk. This is truly a project about making Internet available to everyone, and creating community. It’s not only about each user offering access, but moreover it’s about creating a mesh network with the other members (FON had also this idea at the very beginning, but it never happened and I don’t think this will change)

Getting information (relevant) about freifunk is not easy. The classic community syndrome. Everyone has some information lying somewhere, which becomes outdated. Since they have a community, I simply met some of them lying around in the c-base and they helped me to configure my router without hesitating.

Freifunk uses the ad-hoc mode offered by wifi devices, a connection topology where everyone directly sends the data to the neighbor in range (as opposed to the infrastructure mode, with a central point where every “client” sends the data to). Actually I learned that ad-hoc is not often used (how sad), and very poorly supported by the actual wifi cards.


browsing enhancement

June 2, 2010

Here some browser plugin I would like to see/develop :

  • URL direct redirect :
  • A plugin that convert the URL links that finally point to another site but go through the mwa site for counting purposes.
    ex :

    Interesting : Verify Redirect

  • HTTPS redirect
  • Verifies if the same webpage is available over HTTPS and redirect if yes (also add possibility to redirect to other URL : has secured verion
    Interesting : Redirector

  • MAP subsitute
  • Add the possibility to change the map server within the map boxes. e.g. Have OSM maps when website uses GoogleMap API.
    Interesting : OsmJumper