Those who lead …

Those who lead inspire

Simon Sinek: How great leaders inspire action: TED TALKS

Advertisements
Posted in Uncategorized

Ideas and Execution

1. It’s fairly common to get ideas. The main challenge is to implement them all!

2. What is the most important thing for execution: THE TEAM, passion, commitment, patience and of course skills!

3. Execute it with a long-term vision!

Tagged with:
Posted in Uncategorized

Akamai Timeouts with German Websites and OpenDNS

Since the beginning of this year I get timeouts and very low loading times on multiple German websites,
for example
xing.de
bild.de
bmw.de
audi.de

Other websites works quite well, e.g. telekom.de or heise.de

ISP: Deutsche Telekom (DSL 16Mbit/s)
DNS-Provider: OpenDNS (208.67.222.222,208.67.220.220)

All these problematic websites have one thing in common: they use a CDN, here Akamai.

Example:

http://bild.de

Media_httpbilderbildd_sbgzi

dig @208.67.222.222 bilder.bild.de

;; ANSWER SECTION:
bilder.bild.de.        145    IN    CNAME    bilder.bild.de.edgesuite.net.
bilder.bild.de.edgesuite.net. 20630 IN    CNAME    a122.d.akamai.net.
a122.d.akamai.net.    5    IN    A    77.67.28.74
a122.d.akamai.net.    5    IN    A    77.67.28.91

There are 3 observations that could lead to the problem of timeouts of loading German websites with Deutsche Telekom.

1) OpenDNS requests are routed to NL not to DE

Why are OpenDNS requests routed through Deutsche Telekom to NL and not to DE?
OpenDNS operates server in Frankfurt/DE, but why are they not used?

traceroute to 208.67.222.222 (208.67.222.222), 64 hops max, 52 byte packets
 1  192.168.0.1 (192.168.0.1)  2.826 ms  1.365 ms  1.245 ms
 2  87.186.224.194 (87.186.224.194)  16.368 ms  17.943 ms  17.927 ms
 3  87.186.255.250 (87.186.255.250)  17.241 ms  18.430 ms  17.215 ms
 4  * * 194.25.210.46 (194.25.210.46)  32.941 ms
 5  ae-34-52.ebr2.berlin1.level3.net (4.69.146.153)  31.820 ms  30.188 ms  29.286 ms
 6  ae-3-3.ebr2.dusseldorf1.level3.net (4.69.133.146)  29.429 ms  28.465 ms  31.879 ms
 7  ae-46-46.ebr1.amsterdam1.level3.net (4.69.143.201)  31.685 ms
    ae-48-48.ebr1.amsterdam1.level3.net (4.69.143.209)  31.436 ms
    ae-45-45.ebr1.amsterdam1.level3.net (4.69.143.197)  33.533 ms
 8  ae-59-114.csw1.amsterdam1.level3.net (4.69.153.198)  36.278 ms
    ae-58-113.csw1.amsterdam1.level3.net (4.69.153.194)  32.136 ms
    ae-57-112.csw1.amsterdam1.level3.net (4.69.153.190)  32.282 ms
 9  ae-1-51.edge4.amsterdam1.level3.net (4.69.139.138)  31.865 ms  33.105 ms  31.975 ms
10  splice-comm.edge4.amsterdam1.level3.net (212.72.40.98)  39.748 ms  38.664 ms  40.013 ms

traceroute to 208.67.220.220 (208.67.220.220), 64 hops max, 52 byte packets
 1  192.168.0.1 (192.168.0.1)  5.176 ms  1.489 ms  3.031 ms
 2  87.186.224.194 (87.186.224.194)  16.335 ms  16.928 ms  16.552 ms
 3  87.186.255.242 (87.186.255.242)  16.533 ms  17.211 ms  18.996 ms
 4  194.25.210.46 (194.25.210.46)  30.249 ms  29.390 ms  28.991 ms
 5  ae-34-52.ebr2.berlin1.level3.net (4.69.146.153)  29.144 ms  26.905 ms  26.355 ms
 6  ae-3-3.ebr2.dusseldorf1.level3.net (4.69.133.146)  27.841 ms  28.808 ms  28.599 ms
 7  ae-48-48.ebr1.amsterdam1.level3.net (4.69.143.209)  32.860 ms
    ae-45-45.ebr1.amsterdam1.level3.net (4.69.143.197)  32.144 ms
    ae-46-46.ebr1.amsterdam1.level3.net (4.69.143.201)  30.932 ms
 8  ae-59-114.csw1.amsterdam1.level3.net (4.69.153.198)  32.617 ms  36.026 ms  31.629 ms
 9  ae-1-51.edge4.amsterdam1.level3.net (4.69.139.138)  32.384 ms  30.457 ms  30.320 ms
10  splice-comm.edge4.amsterdam1.level3.net (212.72.40.98)  40.033 ms  40.136 ms  39.262 ms

3) requesting different OpenDNS servers results in different Akamai servers

In my case here through ISP Deutsche Telekom, I get the answer from the NL OpenDNS server to Akamai Server 77.67.28.74 or 77.67.28.91.

#Amsterdam, The Netherlands
#    a122.d.akamai.net
#    bilder.bild.de.edgesuite.net
#    77.67.28.74
#    77.67.28.91
#

traceroute to 77.67.28.91 (77.67.28.91), 64 hops max, 52 byte packets
 1  192.168.0.1 (192.168.0.1)  4.592 ms  1.566 ms  1.381 ms
 2  87.186.224.194 (87.186.224.194)  17.199 ms  16.714 ms  18.311 ms
 3  87.186.255.246 (87.186.255.246)  60.847 ms  16.435 ms  15.820 ms
 4  f-ed3-i.f.de.net.dtag.de (62.154.14.198)  28.209 ms
    194.25.6.62 (194.25.6.62)  27.289 ms
    f-ed3-i.f.de.net.dtag.de (62.154.14.198)  26.678 ms
 5  194.25.211.134 (194.25.211.134)  27.124 ms  27.513 ms  34.102 ms
 6  xe-9-3-0.ams21.ip4.tinet.net (89.149.180.190)  34.250 ms
    xe-5-1-0.ams21.ip4.tinet.net (89.149.185.65)  30.561 ms
    xe-3-0-0.ams21.ip4.tinet.net (89.149.186.130)  32.908 ms
 7  77.67.28.91 (77.67.28.91)  30.941 ms  32.730 ms  31.004 ms

This response I get through other German DNS servers, e.g. Telekom, Hetzner, Vodafone, but not through OpenDNS.

#Frankfurt, Germany   
#    a122.d.akamai.net
#    bilder.bild.de.edgesuite.net
#    77.67.27.57
#    77.67.27.58

traceroute to 77.67.27.57 (77.67.27.57), 64 hops max, 52 byte packets
 1  192.168.0.1 (192.168.0.1)  5.600 ms  1.438 ms  1.159 ms
 2  87.186.224.194 (87.186.224.194)  16.280 ms  16.031 ms  16.660 ms
 3  87.186.255.242 (87.186.255.242)  46.383 ms  19.214 ms  16.272 ms
 4  f-ed3-i.f.de.net.dtag.de (62.154.14.186)  27.957 ms
    194.25.6.26 (194.25.6.26)  26.822 ms
    f-ed3-i.f.de.net.dtag.de (62.154.14.186)  26.968 ms
 5  194.25.211.134 (194.25.211.134)  25.488 ms  26.434 ms  28.173 ms
 6  xe-1-2-1.fra23.ip4.tinet.net (89.149.184.78)  26.193 ms
    xe-4-0-1.fra23.ip4.tinet.net (89.149.181.157)  28.130 ms
    xe-9-0-1.fra23.ip4.tinet.net (89.149.181.205)  26.784 ms
 7  77.67.27.57 (77.67.27.57)  25.829 ms  27.276 ms  25.926 ms

As the NL DNS servers are requested, different Akamai server are responsible for delivery: 77.67.28.74 or 77.67.28.91.

3) Akamai server 77.67.28.91 (NL) ping timeouts

Why are there ping timeouts to one of the NL akamai server through Deutsche Telekom ISP:

PING 77.67.28.91: 56 data bytes
64 bytes from 77.67.28.91: icmp_seq=0 ttl=58 time=43.979 ms
Request timeout for icmp_seq 1
64 bytes from 77.67.28.91: icmp_seq=2 ttl=58 time=37.295 ms
64 bytes from 77.67.28.91: icmp_seq=3 ttl=58 time=34.840 ms
64 bytes from 77.67.28.91: icmp_seq=4 ttl=58 time=32.596 ms
Request timeout for icmp_seq 5
64 bytes from 77.67.28.91: icmp_seq=6 ttl=58 time=36.561 ms
Request timeout for icmp_seq 7
64 bytes from 77.67.28.91: icmp_seq=8 ttl=58 time=36.848 ms
64 bytes from 77.67.28.91: icmp_seq=9 ttl=58 time=33.648 ms
64 bytes from 77.67.28.91: icmp_seq=10 ttl=58 time=34.241 ms
Request timeout for icmp_seq 11
64 bytes from 77.67.28.91: icmp_seq=12 ttl=58 time=37.528 ms
64 bytes from 77.67.28.91: icmp_seq=13 ttl=58 time=36.146 ms
^C
— bilder.bild.de ping statistics —
14 packets transmitted, 10 packets received, 28.6% packet loss
round-trip min/avg/max/stddev = 32.596/36.368/43.979/2.982 ms

traceroute to 77.67.28.91 (77.67.28.91), 64 hops max, 52 byte packets
 1  192.168.0.1 (192.168.0.1)  4.592 ms  1.566 ms  1.381 ms
 2  87.186.224.194 (87.186.224.194)  17.199 ms  16.714 ms  18.311 ms
 3  87.186.255.246 (87.186.255.246)  60.847 ms  16.435 ms  15.820 ms
 4  f-ed3-i.f.de.net.dtag.de (62.154.14.198)  28.209 ms
    194.25.6.62 (194.25.6.62)  27.289 ms
    f-ed3-i.f.de.net.dtag.de (62.154.14.198)  26.678 ms
 5  194.25.211.134 (194.25.211.134)  27.124 ms  27.513 ms  34.102 ms
 6  xe-9-3-0.ams21.ip4.tinet.net (89.149.180.190)  34.250 ms
    xe-5-1-0.ams21.ip4.tinet.net (89.149.185.65)  30.561 ms
    xe-3-0-0.ams21.ip4.tinet.net (89.149.186.130)  32.908 ms
 7  bilder.bild.de (77.67.28.91)  30.941 ms  32.730 ms  31.004 ms

This seems to be the root cause of the problem with accessing big German websites through Deutsche Telekom and OpenDNS.
Maybe it is routing problem from Akamai to Deutsche Telekom or an Akamai (77.67.28.91) problem itself.

The question is now which hostmaster can solve this problem: Akamai, OpenDNS, Deutsche Telekom?

[1] Similar T-online Forum Entry from another user at 2011-12-27 http://foren.t-online.de/foren/read/service/dsl-festnetz/internet-performance…

Tagged with:
Posted in Uncategorized

InfoQ: Guardian.co.uk Switching from Java to Scala

Guardian.co.uk Switching from Java to Scala

Posted by

Charles Humble

on

Apr 04, 2011

Community
Java
Topics

Programming

Tags

lift

,

Scala

,

Apache Solr

The team behind guardian.co.uk which, according to its editor, has the second highest readership of any on-line news site after the New York Times, is gradually switching from Java to Scala, starting with the Content API, which provides a mechanism for selecting and collecting Guardian content.

The guardian.co.uk website comprises about 100,000 lines of code. It uses a fairly typical open-source Java stack of Spring, Apache Velocity and Hibernate with Oracle providing the database. Like the website, the Content API was initially being developed in Java, but the team decided to switch to another JVM-based language, Scala, in its place. Web Platform Development Team Lead Graham Tackley told us

We’ve been a primarily Java development shop for a number of years now, and this has largely served us well. However, as a news website we want to be able to respond to events very quickly. The core Java platform that delivers www.guardian.co.uk has a full release every two weeks. Compared with many enterprise Java applications, this is excellent. Compared with other websites, it’s very poor.

So we’ve been looking for a while at tools, approaches and languages that enable us to deliver functionality faster. This includes using lighter weight Java frameworks like Google Guice, radically different approaches to Java development like the Play framework, and using other platforms such as Python with Django. As part of this exercise we’d been playing with Scala for a while, but unlike the others we hadn’t yet used it for any production code.

We were very keen that the first non-beta release of the Content API (API, Open Platform) should be the first iteration of an ongoing evolving API, which could quickly evolve as we discovered all the interesting use cases that we hadn’t initially thought of. To do this safely without breaking API clients, we needed a comprehensive set of integration tests. After some experimentation of writing these in Java, we decided instead to write just the integration tests in Scala, for three main reasons:

  1. The flexibility of the testing DSL provided by ScalaTest.
  2. We wanted to be excited about writing the integration tests, rather than them being a chore.
  3. Using Scala just for the tests meant we got to use it in anger without impacting production code directly.

After about four weeks of writing just the tests in Scala, we got fed up of having to write the main code in Java, and decided to convert the whole lot to Scala.

InfoQ: In general terms, how did you go about the migration? Did you re-write all the Java code in Scala for instance, or did you combine the two for a while?

The beta version of the Content API was based on a proprietary search engine. The current API uses the excellent Apache Solr (a talk on guardian.co.uk’s use of Solr can be found here), and is also quite different in style to the beta one – the beta did a great job of showing us what we didn’t want the API to look like. Therefore, before Scala came into the picture, we’d decided to re-implement the API rather than reuse the beta codebase.

We’d spent around six weeks with three people implementing in Java before we introduced Scala, so there wasn’t a massive codebase to migrate. However, we weren’t prepared to stop the project for a couple of weeks while we converted to Scala, so we migrated the existing integration tests gradually. As we’d used Maven as a build tool, introducing Scala was a matter of following the instructions to use the maven-scala-plugin to build mixed Java/Scala projects. This allows Java and Scala code to co-exist in the same project, and bi-directionally depend on each other. So we could convert on a class-by-class basis from Java to Scala, which worked far better than we ever imagined: it really did just work.

We took the same approach when converting the main code: over a number of weeks, as we touched a bit of code, we converted it. We then had a couple of days mop up at the end.

InfoQ: What are the libraries/frameworks that you have used for development?

Since we were using a language new to us all, we decided to limit the amount of new stuff that we needed to learn. We chose to stick with plain servlets wired with Google Guice, which is how we build our Java apps now. We use SolrJ, the Java Solr library, to talk to Apache Solr, Joda-Time for date time manipulation and Mockito for unit test mocking (this worked fine with Scala code too).

Sometimes we consciously chose to stick with what we knew to ensure timely delivery: the XML formatted endpoints are generated not using Scala’s excellent XML support, but using javax.xml.stream.XMLStreamWriter just as we would in Java code. We’d already written this before moving to Scala; it worked, it was readable, so we left it. However, we did switch to use the excellent JSON library from Lift – lift-json – to generate the JSON formatted endpoints as the code was far clearer than with the Java JSON library we were using.

InfoQ: What IDEs do you use for development? What is Scala IDE support like?

We use Jetbrains IntelliJ IDEA 10, some of us use the community edition and some use the ultimate edition. The Scala plugin is pretty good but not perfect. Code completion, find usages, and similar navigation nearly always works just fine. It’s not as good as Java at red highlighting code that isn’t valid, and we had some problems with it finding ScalaTest test methods, but other than that we were in our familiar environment working as we always had, just in a much more powerful language.

InfoQ: I’m assuming the majority of the developers on the project were Java programmers? How easy did the developers on the project find learning Scala?

Yes, all of us were quite experienced Java programmers. The initial team of four had huge fun learning Scala: often one of us would come in raving about this new Scala feature we’d discovered and sharing it with the rest of the team. We had a buzz that had long been missing in our Java development. Because we were all learning together, this worked really well. In the first couple of weeks, though, there were occasions when we’d be battling to implement something in a good Scala way and couldn’t figure it out. Knowing you could just churn out the Java code made this particularly frustrating. There were a few days where we went home in frustration saying, "We’re going back to Java tomorrow". Each time, a fresh look in the morning was all it needed to move on.

Since then, we’ve had around ten other Java devs move to pick up Scala. As always, people learn at different speeds and in different ways, but all have come through that and nearly all now get frustrated when they have to write Java code.

One of the things we compare learning Scala against is moving to a different platform like Python/Django or Ruby on Rails. With Scala, at least 75% of what you’re working with is the same as in Java. You can use the same libraries and IDE, the way you package jars and wars is the same, your runtime environment and runtime characteristics are the same. A good Java developer can learn to write Java-style code in Scala in a day, then they learn the power of closures and implicit conversions and very soon they’re more productive than they were in Java.

InfoQ: One of the common criticisms of Scala as a language boils down to it being too complex. A lot of the time I think this is really about readability – the idea being that it is easier to pick up someone else’s code if it is written in a more rigid language like Java. Do you think the criticism is fair? How do you counter it?

I agree, readability is by far the most important characteristic of a codebase. I don’t care whether code is imperative or functional, or is idiomatic Scala or Java-without-semicolons, I only care whether it’s readable. When we were learning new Scala features, we chose whether to use them based on whether the intent of the resulting code was more obvious. In one example, we tried using the Scala Either class to eliminate a few If statements: the team collectively concluded that the If statements were more readable, so we dropped the use of Either in that case.

It’s true that due to the rigidity of Java individual lines of code are always easily understood. But that’s rarely the problem in understanding any non-trivial codebase: I don’t want to understand the detail, I want to understand the intent. Good class design and OO techniques help address this in Java, but I still often find when reading Java code that I cannot see the wood for the trees. In Scala I have the power to express the intent in a way I rarely can in Java.

For example, the Content API needs to decide whether to return results in XML, JSON or redirect to the HTML explorer. We support a format=query string, adding a .xml or .json extension, and specification in an http Accept header. Here’s the code that does that, which I think is a good example of how Scala’s power aids expression of intent (it’s just chaining calls to Scala’s Option class):

great news, looking forward to read more about sites switching from Java to Scale - we still live with Grails.

Posted in Uncategorized

PyCon 2010:Scaling your Python application on EC2 (#191)

There was a presentation that jedberg (one of the guys at Reddit) gave about moving to the cloud. You can watch it here: http://pycon.blip.tv/file/3257303/

Source: http://us.pycon.org/2010/conference/talks/

Tagged with: ,
Posted in Uncategorized

Android and Iphone Privacy

Last week, an Android study [2] by several people from Intel, Penn State and Duke Universties was published that showed how 15 popular applications in the Android Market shared location data with advertisement servers, while 10 shared unique identifiers with them, as well. Additionally Eric Smith (University in Lewisburg, Pennsylvania) published an Iphone privay study [3] on October 1st, 2010 and shows where and how the iOS shares UDIDs with whom, and how the UDIDs are being used.

The iPhone’s UDID is eerily similar to the Pentium 3’s Processor Serial Number (PSN). While the Pentium 3 PSN
elicited a storm of outrage from privacy rights groups over the inherent risks associated with the sharing of such
information with third parties, no such concerns have been raised up to this point regarding the iPhone UDID. As
UDIDs can be readily linked to personally-identifiable information, the “Big Brother” concerns from the Pentium 3
era should be a concern for today’s iPhone users as well.

[1] http://www.engadget.com/2010/09/30/study-select-android-apps-sharing-data-wit…

[2] Android privacy study (2010-09) by William Enck et al.

[3] Iphone study (2010-10) by Eric Smith

Tagged with: ,
Posted in Uncategorized

Instant Google ABC – Germany II 09.09.2010

In my previous post today I collected the Instant Google ABC result for google.com

Here are the today first letter result for the Instant Google ABC on google.de:

a: amazon; ard; aldi 
b: bild; bahn; bahn.de
c: chip; charts; chefkoch
d: das örtliche; deutsche bahn; dhl
e: ebay; ebay.de; edeka
f: facebook; facebook.de; freenet
g: gmx; gmx.de; google maps
h: h&m; hermes; hotmail
i: ikea; icq; itunes
j: jappy; jobbörse; java
k: kino.to; kicker; kalender
l: leo; lidl; lotto
m: meinvz; maps; msn
n: netto; ntv; neckermann
o: otto; o2; obi
p: postbank; paypal, post
q: qvc; quoka; quelle
r: routenplaner; rtl; real
s: svz; saturn; skype
t: telefonbuch; t-online; tv programm
u: ups; uhrzeit; uci
v: vw; vodafone; vlc
w: wetter; web.de; web
x: xing; xbox; xampp
y: youtube; yahoo; youtube.de
z: zdf; zalando; zitate

Who sponsors “java”, “vlc”, “uhrzeit” and “xampp” ?

Tagged with:
Posted in Uncategorized