Notes on the Kafka Summit London 2018

Código fuente

These are some notes and take­aways on the re­cent­ly cel­e­brat­ed Kaf­ka Sum­mit 2018 in Lon­don.

The con­fer­ence was or­ga­nized in three par­al­lel tracks for ses­sions that were cov­er­ing stream pro­cess­ing, pipeli­nes, and in­ter­nal­s. To get a good ex­pe­ri­ence, I at­tend­ed talks of the three type­s, but with a lit­tle pref­er­ence to­wards in­ter­nals and stream­s.

It was a two-­day con­fer­ence with lots of valu­able tech­ni­cal con­tent, awe­some talk­s, speak­er­s, and a lot more. Here are the high­light­s.

Day 1

The Keynotes

The first keynote, “The Death and Re­birth of the Event Driv­en Ar­chi­tec­ture”, was the per­fect way to start the con­fer­ence. To­tal­ly a mind-open­ing talk, to get the right idea of what is the state of the art in terms of dis­trib­uted sys­tem­s, and even­t-­driv­en ar­chi­tec­tures in mod­ern soft­ware en­gi­neer­ing.

Some of the main ideas, phras­es, and con­cepts of this talk:

  • Not just the ar­chi­tec­­tures, but al­­so the busi­­ness are even­t-ori­en­t­ed. You can think a busi­­ness, ev­ery busi­­ness, as a large set of events oc­cur­ring at all times. The ar­chi­tec­­ture then, is just re­flec­t­ing this.

  • Ev­ery busi­­ness is now dig­i­­tal. Start-ups cre­ate their ar­chi­tec­­ture, and build their busi­­ness up­­on it. Lega­­cy com­­pa­nies, adapt to this, so their soft­­ware ar­chi­tec­­ture be­­comes the core of the busi­­ness, and they op­er­ate based on this.

The rest of the talks were amaz­ing as well, and in the ma­jor­i­ty I chose talks of the tracks of stream pro­cess­ing and pipe­lines with some ex­cep­tions on which I at­tend­ed a talk about in­ter­nals that ex­plained key met­rics to mon­i­tor in a Kaf­ka in­stal­la­tion, how Kaf­ka us­es the JVM heap in­ter­nal­s, and why this (a­long with a lot of more met­ric­s, such as the Ker­nel I/O pag­i­na­tion cache) are key met­rics to mon­i­tor.

At the end of the day, I had the chance to talk to some of the speak­ers and keep up en­light­en­ing dis­cus­sions about event sourc­ing ar­chi­tec­tures.

Day 2

Short­er than the pre­vi­ous one (fin­ished at around 1p­m), but the talk were equal­ly amaz­ing.

It was im­pres­sive to see how some big com­pa­nies use Kaf­ka di­rect­ly on Dock­er con­tain­er­s, with Ku­ber­netes, and a mon­i­tor­ing plat­form au­to­mat­i­cal­ly send­ing met­rics to Grafana. There was a talk ex­plain­ing the im­mense in­fra­struc­ture at CERN, and how Kaf­ka plays a cru­cial role on their da­ta pro­cess­ing so­lu­tion­s.

All in all it was a great con­fer­ence, on which I learnt a lot about Kafka, dis­trib­uted sys­tem­s, event sourc­ing ar­chi­tec­ture. Be­sides learn­ing about the tech­nol­o­gy it was great to see how com­pa­nies do re­al-­world im­ple­men­ta­tion of these so­lu­tion­s, and ex­change opin­ions with fel­low pro­fes­sion­al­s.