Creating graphs with Gnuplot… for dummies

As part of an investigation project at work, we had to create a number of graphs. Of course our first idea was using Excel; but it turns out that in a lot of scenarios it’s ambiguous, time consuming and sometimes outright frustrating. So we ended up doing it with Gnuplot, which provided a much better experience.

This article is not meant to give extended coverage of course; there are many FAQs and other documents available online for that (a small collection is given at the end of the article). It’s meant to cover basic usage and some common scenarios, namely:

  • How to download and install
  • How to plot a simple function
  • How to plot data points from a file
  • How to plot multiple functions and/or data points
  • How to setup the plot (axes etc.)
  • How to fill the area between functions
  • How to export the plot for MS Office
  • How to plot using batch files
  • Links and FAQs

Gnuplot is a really powerful tool. This article won’t cover many things, like 3D plots, polar coordinates, binary data, financial-style graphs and others; take a look at the demo library for that (link given at the end).

How to download and install

Download is provided from Sourcefourge. Go to http://www.gnuplot.info/download.html and get the current version.
After downloading, installation is pretty easy and straightforward. Just click “next” in every step and you’ll be ok.

After installation, start Gnuplot from the desktop icon. You’ll get a command prompt (gnuplot>).

How to plot a simple function

A major problem with MS Excel is that you cannot create a graph for a function; you have to create the data in cells, using a formula. And of course, the values will not be continuous but discrete.

So let’s say you want to make a graph of a function f(x)=x^2+10/x, for values of x between -10 and 10. Enter these commands to the command prompt, pressing ENTER after each line (lines that begin with # are comments):

# setup the x axis range
set xrange [-10:10]
# plot our function
f(x)=x**2+10/x
plot f(x)

To change the line color, the easiest way is to use one of the available linestyles:

plot f(x) linestyle 3

 

In order to see the readily available linestyles, just enter:

test

 

How to plot data points from a file

For our example, we have two text tab-separated files, c:\temp\out1.txt and c:\temp\out2.txt, that look like this:

out1.txt

1 30
2 30
3 30
4 30
5 30
10 200
20 200
30 200
40 200
50 200
100 500
200 500
300 500
400 500
500 500
900 500

out2.txt

1 TS 5.3 3.5
2 TS 4.2 7.4
3 TS 6.1 7.3
4 RS 0.9 2.1
5 RS 1.2 2.2
6 RS 0.8 1.9
7 TS 4.9 3.1
8 TS 5.7 2.8
9 TS 4.4 3.4

The first has just two columns, x and y. The second has four columns: the second is labels on x, the third and fourth are measurement values (y) and the first an incremental number (for gnuplot to know which comes first, second etc.)

Let’s plot the first one:

# make sure we're in the correct dir
cd 'c:\temp'
set xrange [0:1000]
set yrange [0:1000]
plot 'out1.txt' using 1:2

Note the 1:2 here. This tells gnuplot that the 1st column of the file will be used for x and the 2nd for y.

If you want to connect the points, the last line would be:

plot 'out1.txt' using 1:2 with lines

If instead of simply connecting the points you would need to do a ‘best fit’ with a given function, say g(x)=a*x+c :

g(x)=a*x+c
fit g(x) 'out1.txt' using 1:2 via a,c
# here you get a list of the calculations gnuplot is doing, the parameters used and the standard error
plot 'out1.txt' using 1:2, g(x)

Of course, that’s not a very accurate fit, but that’s not our point here (smile)

Let’s now plot the second file. Our goal here is to create a bar chart:

cd 'c:\temp'
# 'set autoscale' automatically sets ranges for x,y
set autoscale
set boxwidth 0.5
set style fill solid
plot 'out2.txt' using 1:3:xtic(2) with boxes

Note the 1:3:xtic(2). This tells gnuplot that the 1st column is ot be used for x, the 3rd for y and the 2nd (xtic(2)) for x-axis labels.

Now let’s try to plot two data series in the same bar chart:

cd 'c:\temp'
set style data histogram
set style histogram cluster gap 1
set style fill solid border -1
set boxwidth 0.9p
plot 'out2.txt' using 3:xtic(2) title 'Measurement 14-Feb-2014', 'out2.txt' using 4:xtic(2) title 'Measurement 17-Feb-2014'

How to plot multiple functions and/or data points

Actually we did that already in the example with the fit and bar examples. We just have to give multiple functions/files and separate them with a comma. As an example:

cd 'c:\temp'
a=5
f(x)=a*x
plot f(x), 'out1.txt' using 1:2

Let’s add a line and a legend, shall we ? The last line will become:

plot f(x) title 'My function', 'out1.txt' using 1:2 with line title 'My data'

How to setup the plot (axes etc.)

# Chart title
set title 'Workflow performance (AWTs/sample)'
# Get the legend out of the chart
set key outside
# place the legend
# here you can use 'left', 'right', 'center' and 'top', 'bottom', 'cent'
set key right cent
# Setup the axes range using, e.g.
# From-to
set xrange [1:500]
set yrange [1:1000]
# logarithmic
set logscale x
set logscale y
# axes titles
set xlabel 'Samples'
set ylabel 'AWTs per sample'
# let's see what we've done
replot

How to fill the area between functions

What if you want to fill the area below a curve, or between two curves ?

First, let’s fill the area between a curve f(x)=x^2 and the x axis:

set xrange [0:10]
# c(x) is the same as the x axis
c(x)=0
f(x)=x**2
# '+' is the pseudofile, you can read about it in the documentation
plot '+' using 1:(c($1)):(f($1)) with filledcurves closed linestyle 3 title 'Filled area'

After this, it should be obvious how you can fill the area between two curves; just use a function instead of c(x)=0. Let’s say we use g(x)=x^1.8

set xrange [0:10]
g(x)=x**1.8
f(x)=x**2
plot '+' using 1:(g($1)):(f($1)) with filledcurves closed linestyle 4 title 'Area between two functions'

How to export the plot for MS Office

Although of course you can do a printscreen, the best format to use with Word, Powerpoint etc. is the Enhanced Metafile Format (.emf). The best thing about it is that it’s scalable. Surprisingly, if you have a .emf image and preview it (Windows uses Paint by default) it seems awful; but if you insert it in Word it looks great.

So, in order to create a plot and get it as an .emf you need to do something like this:

# output directory and file name
cd 'c:\temp'
set terminal emf enhanced
set output 'plot.emf'
# create the plot
set xrange [-100:100]
f(x)=x**2-10*x
plot f(x) title 'f(x)=x^2-10x'
# because of 'set output' above, plot creates
# the file on disk instead of showing it on the screen
# NOTE: until you 'unset output', the plot file (.emf, .pdf, whatever)
# is locked and cannot be accessed
unset output
# normally you need to return the plot output to the screen
set terminal wxt

How to plot using batch files

Creating a batch file is useful in several scenarios. For example, you might have a data file (like out1.txt above) which changes every day; and every day you need to create the same graph, but with the fresh data.

So in order to do this, just write all gnuplot commands to a text file and execute it with gnuplot.exe. See the example below (the backslash means that the line is continued):

C:\temp\dailychart.plt
cd 'c:\temp'
set terminal pdf enhanced
set output "plot.pdf"
f(x) = 980/x
c(x) = 650
plot \
f(x) title '980 limit' linestyle 1, \
c(x) title '2 sec limit' linestyle 2, \
'out1.txt' using 1:2 title 'External data' linestyle 3, \
'out2.txt' using 1:3 title 'External data 2' linestyle 4
unset output
quit

The .plt extension is the gnuplot default, but you can name the file anything (e.g. .txt). Now open a Windows command prompt and type:

"C:\Program Files (x86)\gnuplot\bin\gnuplot.exe" c:\temp\dailychart.plt

Links and FAQs

By far the best resource is the demo scripts library. Here you can find almost anything and adapt it for your scenario.

Demo scripts for gnuplot (with images of output)

http://www.gnuplot.info/demo/

gnuplot Quick Reference (.pdf)

http://www.gnuplot.info/docs_4.0/gpcard.pdf

Plotting functions

http://www.gnuplotting.org/plotting-functions/

A Brief Manual and Tutorial

http://people.duke.edu/~hpgavin/gnuplot.html

gnuplot FAQ

http://www.gnuplot.info/faq/faq.html

Official gnuplot documentation

http://www.gnuplot.info/documentation.html

How many coffee capsules is it safe to drink per day?

I’ll concentrate on Dolce Gusto capsules for the simple reason that that’s what I have at home (well, that, plus a Krups filter coffee machine, plus an Izzy traditional espresso machine, plus my one-time favorite Bialetti brikka). The results for Nespresso et. al. should be similar.

dolce-gusto-capsules

Do note that I’m only considering caffeine content; but that’s not always the only factor. E.g. if you drink anything near 400 cups Lungo decaffeinato in a single day, you will have non-caffeine related problems (WC attendance comes readily to mind ! 😊).

So, straight from Nestle’s faq:

How much caffeine is in each of the drinks?

Specialty Coffee :
– Cappuccino : 107 Mg Caffeine
– Cappuccino Skinny : 90 Mg Caffeine
– Latte Macchiato : 85 Mg Caffeine
– Vanilla Latte Macchiato : 83 Mg Caffeine
– Caramel Latte Macchiato : 83 Mg Caffeine
– Latte Macchito Skinny : 83 Mg Caffeine
– Mocha : 45 Mg Caffeine
– Cappuccino Ice : 35 Mg Caffeine

Coffee Drinks :
– Caffe Grande Intenso : 130 Mg Caffeine
– Espresso Intenso : 115 Mg Caffeine
– Grande Mild : 106 Mg Caffeine
– Light Roast : 106 Mg Caffeine
– Medium Roast : 103 Mg Caffeine
– Café au Lait : 92 Mg Caffeine
– Lungo : 89 Mg Caffeine
– Espresso : 80 Mg Caffeine
– Lungo decaffeinato : 1 Mg Caffeine

Non-Coffee Drinks
– Chai Tea Latte : 34 Mg Caffeine
– Nestea Peach Iced Tea : 9 Mg Caffeine

The usual number that’s given as “safe” caffeine per day for adults is 400 Mg (eg. see “Caffeine: How much is too much?” from Mayo Clinic here). So this amounts to:

Mg caffeine per capsule Max capsules per day
Max. caffeine per day: 400
Specialty Coffee :
Cappuccino 107 4
Cappuccino Skinny 90 4
Latte Macchiato 85 5
Vanilla Latte Macchiato 83 5
Caramel Latte Macchiato 83 5
Latte Macchito Skinny 83 5
Mocha 45 9
Cappuccino Ice 35 11
Coffee Drinks :
Caffe Grande Intenso 130 3
Espresso Intenso 115 3
Grande Mild 106 4
Light Roast 106 4
Medium Roast 103 4
Café au Lait 92 4
Lungo 89 4
Espresso 80 5
Lungo decaffeinato 1 400
Non-Coffee Drinks
Chai Tea Latte 34 12
Nestea Peach Iced Tea 9 44

Spreadsheet is here if you want to play with the numbers (you can also comment on it). I’ve rounded to closest integer because nobody makes 3 cups of coffee and then decides to have another 0.8 😊

Enjoy !

cup-of-coffee.jpg

Post-truth, clearly illustrated

That’s a piece of news, tweeted by a random guy, that went viral. It’s false, but it didn’t matter at all.

 

Note that this is NOT a typical fake news case. The “guilty” guy -the one who tweeted the wrong info- actually did some effort to verify if his claim was true. Not that much; but this is totally understandable given that he had, like, 40 followers (basically his friends). And when his tweet went viral and was shared 100s of 1000s of times, he tried to find the truth. When he did, he admitted it, deleted the original tweet and posted the fact that it was false.

 

21fakenews-top-jumbo

 

It didn’t matter.

 

It immediately became, and still is, “proof” for a lot of people, asserting a fact that never happened.

 

I’m becoming increasingly desperate. There really doesn’t look any way out of this mess. People will believe anything if they want to believe it. And through the internet, it’s all too easy to find it. True or not; it doesn’t matter.

 

Good news: first conviction for spreading a hoax

This is really a big deal.

Yesterday (Mon 16-May-2016) a court in northern Greece convicted, for the first time, a journalist/blogger for spreading a hoax.

A hoax is a piece of fake and (usually) emotionally charged news item. The usual drivers behind this is “like farming” (earning a small amount of money for every ‘click’ via Google ads) and selling bogus “health” products on the side. It’s very common for hoaxes to go hand in hand with conspiracy theories, like “chemtrails” (“we are being spreyed with chemicals from airplanes!”) or, as in this case, “harmful vaccines” (“vaccines cause autism”, “pharma companies spread cancer through vaccines!”).

Until now, the economics were firmly on the side of the scammers propagating the hoaxes: there was only profit to make, no real cost and, more importantly, no risk. So they would (and are) spreading whatever b*****t they can think of, with no or fake proof but lots of emotional content (“cancer to children!!!”) and pocket the profits.

The hoax of this specific case was titled “Shock: See how companies are spreading cancer through a vaccine”. It was about a girl which is not named other than by first name who supposedly received the MMR vaccine and then died from a brain tumor.

The story is full of sh*t. It was very well researched here.

This conviction is the only one I’m aware of globally (I do hope there are more, but I haven’t heard of any). And it may be, however slowly, a turning of the tide. Organized society needs to fight against this, and such cases are long overdue.

More info here:
http://www.ekathimerini.com/208703/article/ekathimerini/news/court-convicts-hoax-journalist-for-reproducing-false-news

Ιδεοληψία

Μόλις διάβασα ένα άρθρο του BBC σχετικά με τον διάσημο κατάσκοπο-προδότη της Αγγλίας, τον Kim Philby. ​Είναι τρομακτικό να διαβάζεις πόση δύναμη έχει, και που μπορεί να οδηγήσει, η ιδεοληψία:

“[Kim Philby’s] KGB handler next instructs him to get the top job by removing his boss, Felix Cowgill.”

“It was a very dirty story – but after all our work does imply getting dirty hands from time to time but we do it for a cause that is not dirty in any way,” Philby explains.

“I have to admit that was the most blatant intrigue against a man I rather liked and I admired but the instructions stood and nothing I could do would alter them.”

Για όσους ξέρουν ποιός είναι ο Kim Philby ή έχουν διαβάσει έστω και ένα μυθιστόρημα του John le Carré, το άρθρο του BBC είναι must.

http://www.bbc.com/news/uk-35943428

Το τίμημα του Frankenschock… δεν είναι αυτό που νομίζετε

Ακόμα και σε ένα έγκριτο μέσο, όπως η Ναυτεμπορική, βρίσκεται που και που δημοσιογράφος να γράφει για θέμα με το οποίο η σχέση του είναι… πλατωνική:

“Λαμβάνοντας υπ’ όψιν τις οικονομικές και γεωπολιτικές εξελίξεις, και έχοντας επίγνωση των νέων μέτρων νομισματικής στήριξης και του «πακτωλού» ρευστότητας που προετοίμαζε η ΕΚΤ -πρόγραμμα ενός τρισ. ευρώ- η ελβετική κεντρική τράπεζα προτίμησε την τακτική της υποχώρησης στον συναλλαγματικό πόλεμο που είχε ξεσπάσει τότε, αντί την τακτική της άμυνας, γνωρίζοντας πολύ καλά ότι η υπεράσπιση του ανώτατου ορίου με αλλεπάλληλες παρεμβάσεις θα οδηγούσε με μαθηματική ακρίβεια στην «εξαφάνιση» των συναλλαγματικών της διαθεσίμων.”

http://www.naftemporiki.gr/story/1055258/to-timima-tou-frankenschock

(η έμφαση δική μου)

Στην πραγματικότητα, όπως γνωρίζουν οι πάντες, θα συμβεί το ακριβώς αντίθετο ! Το έχουν επισημάνει πολλές φορές, μεταξύ άλλων, οι Financial Times (πχ. “The Swiss hill that may become a mountain“).

Απλή λογική: όταν μια κεντρική τράπεζα θέλεις να στηρίξει (ανεβάσει) την τιμή του νομίσματος, αγοράζει το νόμισμα ξοδεύοντας συνάλλαγμα –συνήθως δολλάρια. Έτσι τα αποθέματα μπορεί να τελειώσουν κάποια στιγμή. Όταν όμως θέλει να ρίξει την τιμή, όπως η ελβετική κεντρική τράπεζα, τότε “τυπώνει” (στην πραγματικότητα πιστώνει ηλεκτρονικά λογαριασμούς τραπέζης) το δικό της νόμισμα. Και αυτό δεν “τελειώνει” ποτέ.

Αυτό δεν σημαίνει ότι η ενέργεια αυτή δεν έχει προβλήματα (όπως εξηγούν οι FT) ή ότι μπορεί να συνεχίζεται επ’ άπειρον. Αλλά οπωσδήποτε δεν θα “εξαφανιστούν τα συναλλαγματικά αποθέματα”, όπως διατείνεται ο συντάκτης της Ναυτεμπορικής.

Serious but hilarious NSA anecdotes

(This one’s for IT guys, perticularly crypto geeks, source is Schneier’s blog)

NSA recently declassified a lectures book from 1973. It contains some real gems, such as these from pages 55/56:

KAG-1/SEC used to be the bible of US cryptographers, was held in every crypto-center and covered everything from message preparation to compromise reporting in considerable detail. While we viewed it as a model of clarity, this perception was not always shared in the real world. A frustrated Navy Chief stormed out of bis crypto-ccntcr on board a carrier at sea, banded KAG-1 to a sailor and jokingly said “Throw this dam’ thing overboard.” He did. Several ships thereafter steamed back and forth for several days, but never found it. Winds, tides, and currents were studied to predict where it might come ashore with results so ambitious as to offer little hope and, in fact, it was never recovered – at least by us.

This incident triggered an R 1 study on what happens to our documents in salt water. A tank was made, and a copy of KAG-1 immersed. It stayed there for a year or so with no sign of deterioration. Agitators were added to stimulate wave action for another few months, with still no appreciable effect. We never did find out how long such a document would last. Subsequent work, however, has shown that good paper is nearly impervious to salt water, apparently indefinitely. A visit to S2’s exhibit of materials recovered from the sea bottom will bear that out. There you can see perfectly legible codes that had been under water since World War II, together with extraordinarily well-preserved items of hardware and magnetic tape that had been on the bottom for many years. These facts add to the previously expressed skepticism about jettison as a way to get rid of our stuff unless at very great depths and in completely secret location. (Shortly after WWII, small Army training crypto-devices called the SIGFOY were disposed of beyond the 100 fathom curve off Norfolk. Some years later, they became prize souvenirs for beach combers as they began washing ashore.)

UNSOLVED PUZZLE – We used to store a lot of cryptomaterial in a warehouse at Ft. Holabird. It was fenced and protected by a 24-hour armed civilian guard. One evening, such a guard saw an individual inside the fence, evidently attempting to penetrate the warehouse. He drew his weapon, cried “Halt!” and led the individual to the guard shack and started to call in for help. About that time, the intruder started running, climbed the fence, and disappeared.

We asked the guard why he didn’t shoot – he said he was afraid he might hurt somebody.

CONFETTI – When we manufacture one-time tape, a by-product of the punching process is millions upon millions of tiny, perfectly circular pieces of paper called “chad” that come out of holes in the tape. This chad was collected in burn bags and disposed of. Someone thought it would make good public relations to give this stuff to high school kids for use as confetti at football games. Inevitably, one of the burn bags was not quite empty when the chad went in. At the bottom, were a couple of TOP SECRET key card book covers and a few assorted keys. They carried the impressive caveats of those days like “CRYPTO – CRYPTO-CLEARANCE REQUIRED” and were, to use a term earlier referred to, “fascinating” to the kids when they discovered them.

One of the girls, whose father happened to be an Army officer, tacked soine of this material on her souvenir board. When Daddy saw it, he spiralled upward. He decided that it must be destroyed immediately; but first made a photograph of it for the record. He tore it up, flushed it away, and reported in. With some difficulty, various cheerleaders and other students who had glommed on to some of this material were tracked down, and persuaded to part with it.

We no lonser issue confetti.

A History of U.S. Communications Security (Volumes I and II);
the David G. Boak Lectures, National Security Agency (NSA), 1973
http://www.governmentattic.org/18docs/Hist_US_COMSEC_Boak_NSA_1973u.pdf