Rome day 4 Ancient Rome

Rush Hour

The planned trip to ancient rome had been delayed due to the very pleasant company of Juliets relatives who live here. With this being the last full day of the holiday - we made a plan to get out early, alas the hotel shuttle was already fully booked so we would start the day off with a ride on the Roma-North line.

The train journey was in rush hour - which initially was quite a placid affair. However as we moved to the Metro - the volume, velocity and veracitiy increased dramatically. It was not London busy - but Rome busy - a more polite way of rushing to work.

The Colleseum

Arriving at 08:45 - we were pleasantly surprised to find not too many annoying "skip the queue" salesmen, possible as the queue for tickets that you buy today was only about 20 deep on the outside.

A quick bag check (alas common on most tourist spots in Rome at the moment), and we were inside standing under the very large arches - waiting to buy a ticket. The line progressed smoothy - and it cost 12 EU each to gain access to the Collesem and the Palintine Hill complex - valid for 48 hours. The Audio guide was another 5 EU.


We stood in the dark trying to figure out how the audio device worked - and once mastered listened to a story about the Colleseum. We then followed the signs, and climbed up a VERY steep set of stairs - to the First floor.

Here there were some explanations of things you have seen whilst watching Gladator..


The guides were all telling the tourists stories of what the common person would be doing at the games, as well as those better off. It did seem an amazingly well organized place - and all before Internet, E-mail and instant messaging !!


As we progressed through the exhibition we could get glimpses of the stage in the middle. But still it's full size and extent was hidden.


Then with little warning you get to see the areana.


It is quite amazing - It was able to seat 150,000 people !! At one point there were 10,000 people fighing in the area !!!

We stopped looked, listened and we were entertained - so the Colleseum get's the thumbs up.

Palentine Hill

Just next to the Colleseum is the start of the Ancient Rome area, it can be best viewed in this next image.


On the left there is Constantine's Arch - the columns (Temple) in the middle lead you towords the Arch of Titus.

As we approached this there was a massive queue - but I was unsure if our tickets were for here as well as the Colleseum - suspecting some clever "No not this place" scam/dodge. So Juliet and I were trying to get to the head of the queue - to confirm we could enter. When just a the right time - a Police car came out - forcing the queuers to scatter - when they reformed amazingly we were very close to the front !!! For once our queueing skills were better than most.

The tickets did work - eventually !!! And we entered the most amazing collection of ruins - possible more interesting than the Colleseum - simply as they were more varied.

Images coming are for Arch of Septimius, Basilica of Julia (I kid you not)


Rome Day 3 - Churches and exploring

The extra long lie in was very much appreciated, and when Juliet's cousin Marlon arrived exactly on time - surely this was a good Omen for the day ahead.

We zoomed around to his appartment - which was ironically very close to out hotel, and his wife and daughter - Maria and Joyce also joined us.

The first port of call was Villa Borgese - where there is an art museum as well as a lovely park. Alas the art museum was fully booked, which meant instead a pleasant walk around the park.


Funnyy fountain face


Lovely fountain in the woods

With ever increasing drizzle - we headed off to someplace indoors - which in Rome I think nearly always means a church. Not just any church - the 4th largest in the world apprantly. Specifically the Basilica of Santa Maria Maggiore.

Basilica of Santa Maria Maggiore

It hardly seems fair to compare any place with the Vatican in terms of grandeur or sheer 'wow', but this Basilica does compete. A first it seems a little strange as to why ... until then you realize (someone tells you) that this church is where the Pope also sometime attends - hence the somewhat lavish decorations.

The results are impressive.


Impressive nave


After walking around here for a while we moved on... yes to another church.

Basilica Of St. John Lateran

From the outside this was even more impressive. A massive gothic building - build just inside the city walls. For those of you who have been to Manila/York - the city walls in parts of Rome just look the same.

Inside was a massive building - but unlike the previous Basilica it was relatively bare with pictures/gold/ornements.


What we did find was a lovely cloister area - the last one I had recently been in was in Cebu Catherderal - this was slightly larger, and with a fantasic collection of Ancient Roman and more modern (nothing after 1700 !!) pieces of stonework.


Pinoy Party

All of this walking around churches is hungry work - Marlon very kindly invited us to a Filipino community party. In the Philippines at this time of year it is Fiesta time - and the Filipinos in Rome have a great solution to missing the fun that is going on back at home - they hold their own fiesta.

Each week the community seems to organise a set of events - it really did feel like we were back in Manila or in this case Pampanga.

A quick look at the food selection - the ladies were not happy as I took this at the end of the meal - there had been much much more food !!!

As they say - "It's more fun in the Philippines"


Rome Day 2 - The Vatican

Feeling pretty good after more than 24 hours with no sleep, the morning after a long flight can often be "please let me sleep" rather than "I want to go and explore the city". Happily today was the latter ...

/galleries/2015/Rome/Day2/h1.thumbnail.jpg /galleries/2015/Rome/Day2/h2.thumbnail.jpg

The Hotel (The Duke Hotel) has an amazing breakfast spread, even more amazing if you have been living in the Middle East for a while - as it is 60% pork based. So after a few plates of this and that - oh and just one more thing.... we were ready to go. It was only 0800 and the free Hotel shuttle bus was going to drop us close to a Metro station.


The previous day's learning expierience on public transport meant we were almost instantly getting off the metro close to the Vatican Museum.

You did not have to ask yourself are we close. The number of tourists alone told us that we were close. The Walls of the Vatican mean that you can only see the Vatican's walls - but on turning toward St Peters square - the line of columns starts to become visible and then almost un-expectadly you are standing on the edge of St Peters square.


St Peters square looking quiet.


Juliet in front of the Vatican.


The reason of arriving quite early was to beat the long queues... but we were already standing about 1/3 around the circumference of the square (the Square is in fact an Oval).

The line moved quickly and we observed some most excellent queue jumping techniques... some of the italians were v v good in this.

After an airport security scan, we then were free to enter the Church (it is not a Cathederal) and to stare with awe at the impressive building, with it's many lovely works of art inside.

Juliet checked what time the next mass was, and we organised our time so that we were able to attend; There was obviously some important function about to happen - Pentecost it turns out, and the Pope was going to be leading the service. The TV crews and chairs were being set up.

Having finished with the Vatican we then started a leisurly stroll towards the river - initially planning on seeing a Castle. This walk was interupted when the ever darkening skies yielded rain. So we took refuge in one of the many restaurants that abund here - only to bumb into a small group of Filipino's who were also visiting the city.


After a pleaant lunch, we continued to the Castle, only to find a long line outside, which did not seem to be moving, so we continuted our walk arriving at the National War memorial about an hour later.


Here we had a rest - whilst looking at the ruins of ancient Rome - before continuing on to the Colleseum. The increase in tourists and people dressed as Ancient Romans was now exponential.


In rome there are lots of tourists - all of who want to see the sites - so you get lots of queues. You also get companies that specialize in selling "jump the queue" tickets - and with any business that "sells tickets" you get street touts - which are both funny and amusing in equal amounts.

Content with seeing Ancient rome (we will be going there for a day out soon) we headed rather weirly back to the hotel (stopping for a quick cafe break).

We had thought that this would have been the end of the day, but Juliet's cousin happens to live in Rome !! So we went out to meet him and his family - and spent several pleasant hours at their appartment close by.

To see all the pictures please go to... here

Welcome to Rome


The flight from Muscat to Dubai was smooth and quick. Dubai looked very impressibe in the early evening.


The 6 hour stop over was also just about passable in the most excellent Terminal 1 - and at 0330 our 777-300 lifted off to Italy.


A faily quiet flight - in which I managed to get some sleep - which for me is a very pleasant change.

Touchdown in Rome - there must have been an immigration queue of at least 2,000 people. Juliet having expierienced this the year before in Milan - quickly started moving through the queue. I was spared this, as the EU queue had only 20 people in it. Instead I waited 1 hour for the baggage to arrive.

Buying train tickets for E14 each (1st class) we were soon shivering on the platform - as the weather here is not like Middle East weather (I am glad to say). On time we departed, and arrived on time. A very nice way to get into the City.

We have just arrived at the hotel - and after 20 hours of travelling we are waiting for the room to be released.


The Hotel is called '''The Duke''' - it is not in a very central location - but it is very pleasantly furnished and staffed. So far we have been very impressed indeed.

After a few minutes rest (ok maybe 50) we headed out to get our bearings, and probptly got lost. To try and make up some lost time - we bought a day's travel pass and jumped on a tram - only to get more lost.


All Rome trains seem to be heavily vandalised both inside and outside.

Eventually we sorted things out and we had a pleasant afternoon seeing some of the sights in rather chilly weather.

A few more pics

Gallery pics here

Philippine Diving Dec 2014



Possibly the best dive for seeing Turtles I have ever been on. This was at 'Tongo Point' about 2 kms from Moalboal.


Just off the shore in Moalboal town 12 years ago - I was lucky to see a large fish ball. It obviously is quite a common thing in this area - as it seems to be always there. Twelve years and underwater photo technoloy has come a long way....

See the amazing bait-ball (literally 80m off the shore from the main bar/restaurant area in Moalnoal town).

Moal muck diving

Although not as famous by any means as Domuguete, the muck diving here was not at all. Of course not being afraid of rather large snakes also helps.

Domaguete muck diving

Possibly the #1 place in the world for muck-diving.

This was the beach directly outside our hotel.


We walked into the sea - finned for 20 meters over flat dull sand, and then went down a quite steep sandy slope.

Apo Island diving

Often talked about - and requiring a day at sea (which is even better) - I had a nice day at Apo. However turning up on your own, joining a group that has been together for several days is never a good start for a days diving.

At least I may see some Dolphins and Turtles.

Xmas EVe 2014

A Wonderful fun eve - best told in picture format.

Gallery pics here

Fuzzing data with Python

Test Data

Gosh - I think I fell asleep... I tried to write about Test Data !!!

I am presently working on a project that requires some extensive testing. So what I hear you mutter, well this project requires data that vaies in it's quality.

I have verified that it works when given good data - but what happens when there is missing data..... and incorrect data.

Helpful site

I came across a good site which has an interesting data generator - I need people type data.... and this site seems to be able to generate just what I need.

As I am mean (and my employer is meaner) I can only get data in 5K chunks. But I can run the command several times per day.....

By appending the output to a file - I can build up records quite quickly. But there is an issue due to the ID number being restarted.

If you use this little 1 line command - this fixes that issue.

cat basic.csv | cut -d ',' -f 2- | nl -n ln  | sed -e 's/[[:space:]]\+/,/g' -e 's/^1,/id,/' > people.csv

The first 20 lines of the people.csv looks something like this....

id,first_name,last_name,email,country,ip_address,City,Country,CCard,Race,Company 2,Janice,Gilbert,,Russia,,Pyt-Yakh,Lebanon,3577598273371963,Melanesian,Yakijo 3,Mary,Adams,,China,,Lücheng,Portugal,3586021087796977,Black,or,African,American,Jamia 4,Frank,Lopez,,Belarus,,Dashkawka,United,States,5602233033286350,Pakistani,Vinte 5,Marie,Martin,,Russia,,Boguchar,China,5411850139268049,Blackfeet,Twimbo 6,Joyce,Price,,Sweden,,Horred,Mongolia,5100176874230168,Creek,Skippad 7,William,Riley,,Colombia,,Guapi,Indonesia,374622899402654,Navajo,Yadel 8,Rose,Cunningham,,Colombia,,Buenaventura,Indonesia,3581970954021361,Indonesian,Gigaclub 9,Beverly,Kim,,China,,Meilisi,Philippines,5602237467769419,Apache,Jatri 10,Lois,Coleman,,China,,Zhihe,Jamaica,5577254622341886,Seminole,Twimm

So where is the missing data ?

I do not just need missing/bad data - I also need good data (so I can test how accuratly the program is working).... so I thought - Lets make an obfuscator.... but one that I can control.

fuzz my fuzzy friend

This is my small module to "fuzz" the data in a controlled yet flexible manner.

__author__ = 'tim'

from optparse import OptionParser
import random

parser = OptionParser()
parser.add_option("-f", "--file", dest="filename",
                  help="file to read", metavar="FILE")
parser.add_option("-d", "--data", type=str, dest="fields", help="data fields to extract i.e. 2,3,4")
parser.add_option("-l", "--loss", type=str, dest="loss",
                  help="percentage of data to loose per output field, i.e. 50,20,90")
(options, args) = parser.parse_args()
loss = [int(a) for a in options.loss.split(',')]
fields = [int(a) for a in options.fields.split(',')]
first_line = True

    ifp = open(options.filename)
    for line in ifp:
        data_line = line.split(',')
        pos = 0
        new_line = ''
        for wanted in fields:
            data = data_line[wanted -1]
            rnd = random.random()
            if pos > 0:
                new_line += ","
            if rnd*100 < loss[pos] or first_line:
                new_line += data
            pos += 1
        first_line = False
except FileNotFoundError:
    print("File %s not found", options.filename)


Using the people.csv file shown earlier.... if

You want the first 3 columns with 0% data loss

python -f people.csv -d 1,2,3 -l 0,0,0

You want fields 1,4,5 with 0% Loss, 50% loss and 70% loss

python -f people.csv -d 1,4,5 -l 0,50,70


Please feel free to use/mod/hack this code as you want. I may add some extra checks (number of fields is valid).

Next task....

Write a scoring script .....

Python logging 2

My python logging has improved much recently (see previous post) - however I think there are more improvements still

Try this

For a better/clearer log try this.....

import logging
# create logger with 'spam_application'
logging.basicConfig(format="%(asctime)s - File: %(filename)s - Line: %(lineno)d - %(message)s",level=logging.INFO)
logger = logging.getLogger(__name__)

Python logging

Just been reworking some code for a friend of mine - which needs to have quite a large amount of logging info. This I have rather painfully been doing in a PERL-eque way.... which in Python looks horrible.

Enter Logging


Its been around for ever .... but I have never used it (hangs head in shame).....

Use it !!

import logging
__author__ = 'tim'

if __name__ == "__main__":
  logger = logging.getLogger(__name__)

If you are using Sub-classes ....

Then place this at the head of the sub/class

import logging
logger = logging.getLogger(__name__)

And then log away....

Log Files/roll-over

All possible - just change the handler in the logger initialization.

Python Dictionary to Pandas data frame

import pandas as pd

I have a dictionary with seconds - and then words I have seem

We need to move them into pandas.... but How ?

I will try directly using from_dict


123 345
0 the little
1 cat boy
2 sat blue
3 on blew
4 the his
5 mat horn

Well this is a start.... but not what I am really looking for.

#Lets make a tuple - and store in an array
for k in word_freq:
    for w in words:
[(345, 'little'),
 (345, 'boy'),
 (345, 'blue'),
 (345, 'blew'),
 (345, 'his'),
 (345, 'horn'),
 (123, 'the'),
 (123, 'cat'),
 (123, 'sat'),
 (123, 'on'),
 (123, 'the'),
 (123, 'mat')]
period word freq
0 345 little 1
1 345 boy 1
2 345 blue 1
3 345 blew 1
4 345 his 1

It's getting there......

word freq
345 little 1
345 boy 1
345 blue 1
345 blew 1
345 his 1
345 horn 1
123 the 1
123 cat 1
123 sat 1
123 on 1
123 the 1
123 mat 1

And now my dataframe looks how I need it....

word blew blue boy cat his horn little mat on sat the
123 0 0 0 1 0 0 0 1 1 1 1
345 1 1 1 0 1 1 1 0 0 0 0

It would be nicer - if the columns were sorted .... so

word blew blue boy cat his horn little mat on sat the
123 0 0 0 1 0 0 0 1 1 1 1
345 1 1 1 0 1 1 1 0 0 0 0

I hope that you found this useful

Pandas is a very flexible and useful module

Pandas Example Data Frames

import pandas as pd
import numpy as np
ser = pd.Series(np.arange(10, 16))
df = pd.DataFrame(np.arange(30).reshape(6, 5), columns=list('abcde'))

mul_df =pd.DataFrame(np.arange(12).reshape((4, 3)),
                index=[['a', 'a', 'b', 'b'],
                       [1, 2, 1, 2]],
                 columns=[['Ohio', 'Ohio', 'Colorado'],
                            ['Green', 'Red', 'Green']])

mul_df.index.names = ['let', 'num']

mul_df.columns.names = ['state', 'color']

ts_df = pd.util.testing.makeTimeDataFrame(nper=500)

big_df = pd.DataFrame(np.random.randint(500, size=(500, 10)),
pdr=pd.date_range(start='1-1-2000', end='12/31/2012', freq='Q')
2000-01-03 0.358643 -0.517784 2.386702 -0.772061
2000-01-04 1.346614 -0.437451 -1.792780 -1.215079
2000-01-05 0.505089 0.624540 0.016831 -1.376099
2000-01-06 -0.283964 -0.470478 -2.337222 0.025092
2000-01-07 1.421577 -1.576425 -0.855934 -0.205375
2000-01-31 -0.229302 0.181979 -0.003850 -0.324731
2000-02-29 -0.148118 -0.088294 0.287306 -0.145153
2000-03-31 0.297684 -0.375921 -0.085199 -0.251943
2000-04-30 -0.027088 -0.052444 0.110929 0.192342
2000-05-31 0.189805 -0.287204 -0.303970 0.000853
2000-06-30 -0.208958 0.194892 0.311703 0.233887
2000-07-31 0.174342 -0.239666 0.089972 -0.393657
2000-08-31 0.037135 -0.118889 -0.080718 -0.454320
2000-09-30 -0.235596 -0.115370 0.173912 -0.206706
2000-10-31 -0.196163 -0.016557 0.478976 0.143683
2000-11-30 -0.199093 -0.045857 -0.297545 -0.183729
2000-12-31 -0.054346 0.356467 -0.117867 0.178705
2001-01-31 -0.187821 0.444874 0.639559 -0.093225
2001-02-28 -0.184460 -0.031779 -0.071690 0.067420
2001-03-31 0.228944 -0.031333 0.250906 0.015220
2001-04-30 0.245102 0.219844 -0.095021 -0.288760
2001-05-31 -0.245239 0.004290 -0.417796 0.291021
2001-06-30 -0.027298 0.074514 0.025442 0.134928
2001-07-31 -0.007028 0.161735 0.014365 0.233097
2001-08-31 -0.154303 0.308349 0.036484 0.078787
2001-09-30 -0.331431 -0.588452 0.033369 -0.436809
2001-10-31 0.269592 0.167445 0.421319 -0.171788
2001-11-30 -0.068738 -0.230188 0.012225 -0.324455
ts_df.resample('2M', how='first')
2000-01-31 0.358643 -0.517784 2.386702 -0.772061
2000-03-31 -0.799455 1.351663 1.188046 -1.424432
2000-05-31 -0.984423 -1.501314 -0.566439 1.476801
2000-07-31 -0.665194 -0.207105 0.072863 0.434457
2000-09-30 -0.166712 -0.208606 0.289842 0.200681
2000-11-30 1.659910 -1.158376 0.987879 -0.717845
2001-01-31 0.078693 1.539083 -0.100980 -1.626005
2001-03-31 0.063028 0.585999 -0.825291 0.524521
2001-05-31 0.205192 -0.165846 -0.029899 0.852832
2001-07-31 -1.592271 0.769258 -0.817421 1.128302
2001-09-30 -0.014189 -1.697985 -0.497237 -0.021580
2001-11-30 -1.841321 1.756838 1.719043 -0.353030
ts_df.resample('2H', how='first')
2000-01-03 00:00:00 0.358643 -0.517784 2.386702 -0.772061
2000-01-03 02:00:00 NaN NaN NaN NaN
2000-01-03 04:00:00 NaN NaN NaN NaN
2000-01-03 06:00:00 NaN NaN NaN NaN
2000-01-03 08:00:00 NaN NaN NaN NaN
2000-01-03 10:00:00 NaN NaN NaN NaN
2000-01-03 12:00:00 NaN NaN NaN NaN
2000-01-03 14:00:00 NaN NaN NaN NaN
2000-01-03 16:00:00 NaN NaN NaN NaN
2000-01-03 18:00:00 NaN NaN NaN NaN
2000-01-03 20:00:00 NaN NaN NaN NaN
2000-01-03 22:00:00 NaN NaN NaN NaN
2000-01-04 00:00:00 1.346614 -0.437451 -1.792780 -1.215079
2000-01-04 02:00:00 NaN NaN NaN NaN
2000-01-04 04:00:00 NaN NaN NaN NaN
2000-01-04 06:00:00 NaN NaN NaN NaN
2000-01-04 08:00:00 NaN NaN NaN NaN
2000-01-04 10:00:00 NaN NaN NaN NaN
2000-01-04 12:00:00 NaN NaN NaN NaN
2000-01-04 14:00:00 NaN NaN NaN NaN
2000-01-04 16:00:00 NaN NaN NaN NaN
2000-01-04 18:00:00 NaN NaN NaN NaN
2000-01-04 20:00:00 NaN NaN NaN NaN
2000-01-04 22:00:00 NaN NaN NaN NaN
2000-01-05 00:00:00 0.505089 0.624540 0.016831 -1.376099
2000-01-05 02:00:00 NaN NaN NaN NaN
2000-01-05 04:00:00 NaN NaN NaN NaN
2000-01-05 06:00:00 NaN NaN NaN NaN
2000-01-05 08:00:00 NaN NaN NaN NaN
2000-01-05 10:00:00 NaN NaN NaN NaN
... ... ... ... ...
2001-11-27 14:00:00 NaN NaN NaN NaN
2001-11-27 16:00:00 NaN NaN NaN NaN
2001-11-27 18:00:00 NaN NaN NaN NaN
2001-11-27 20:00:00 NaN NaN NaN NaN
2001-11-27 22:00:00 NaN NaN NaN NaN
2001-11-28 00:00:00 -0.076379 -1.051271 0.259663 0.207786
2001-11-28 02:00:00 NaN NaN NaN NaN
2001-11-28 04:00:00 NaN NaN NaN NaN
2001-11-28 06:00:00 NaN NaN NaN NaN
2001-11-28 08:00:00 NaN NaN NaN NaN
2001-11-28 10:00:00 NaN NaN NaN NaN
2001-11-28 12:00:00 NaN NaN NaN NaN
2001-11-28 14:00:00 NaN NaN NaN NaN
2001-11-28 16:00:00 NaN NaN NaN NaN
2001-11-28 18:00:00 NaN NaN NaN NaN
2001-11-28 20:00:00 NaN NaN NaN NaN
2001-11-28 22:00:00 NaN NaN NaN NaN
2001-11-29 00:00:00 0.237788 0.841239 -0.132461 0.714544
2001-11-29 02:00:00 NaN NaN NaN NaN
2001-11-29 04:00:00 NaN NaN NaN NaN
2001-11-29 06:00:00 NaN NaN NaN NaN
2001-11-29 08:00:00 NaN NaN NaN NaN
2001-11-29 10:00:00 NaN NaN NaN NaN
2001-11-29 12:00:00 NaN NaN NaN NaN
2001-11-29 14:00:00 NaN NaN NaN NaN
2001-11-29 16:00:00 NaN NaN NaN NaN
2001-11-29 18:00:00 NaN NaN NaN NaN
2001-11-29 20:00:00 NaN NaN NaN NaN
2001-11-29 22:00:00 NaN NaN NaN NaN
2001-11-30 00:00:00 0.627325 -1.253637 -1.082497 0.732089

8365 rows × 4 columns