Notes on the Kafka Summit London 2018

The­se are so­me no­tes and takeawa­ys on the re­cen­tly ce­le­bra­ted Ka­fka Su­m­mi­t 2018 in Lon­don.

The con­fe­ren­ce was or­ga­ni­zed in th­ree pa­ra­llel tra­cks for ses­sions that we­re ­co­ve­ring stream pro­ce­s­sin­g, pi­pe­li­nes, and in­ter­nal­s. To get a good ex­pe­rien­ce, I atten­ded ta­lks of the th­ree ty­pes, but wi­th a li­ttle pre­fe­ren­ce to­war­d­s in­ter­nals and strea­ms.

It was a two­-­day con­fe­ren­ce wi­th lo­ts of va­lua­ble te­ch­ni­cal con­ten­t, awe­so­me ­ta­lks, speaker­s, and a lot mo­re. He­re are the hi­gh­li­gh­ts.

Day 1

The Keynotes

The first ke­y­no­te, “The Dea­th and Re­bir­th of the Event Dri­ven Ar­chi­tec­tu­re”, was the per­fect way to start the con­fe­ren­ce. To­ta­lly a min­d-o­pen­ing ta­lk, to­ ­get the ri­ght idea of what is the sta­te of the art in ter­ms of dis­tri­bute­d s­ys­te­ms, and even­t-­dri­ven ar­chi­tec­tu­res in mo­dern so­ftwa­re en­gi­nee­rin­g.

So­me of the main idea­s, ph­ra­ses, and con­cep­ts of this ta­lk:

  • Not just the ar­­chi­­te­c­­tu­­res, but al­­so the bu­­si­­ness are even­­t-o­­­rien­­te­­d. You can ­­thi­nk a bu­­si­­ness, eve­­ry bu­­si­­ness, as a la­r­­ge set of even­­ts oc­­cu­­rring at all ­­ti­­me­s. The ar­­chi­­te­c­­tu­­re then, is just re­­fle­c­­ting this.

  • Eve­­ry bu­­si­­ness is now di­­gi­­ta­­l. Sta­r­­t-ups crea­­te their ar­­chi­­te­c­­tu­­re, and bui­l­­d ­­their bu­­si­­ness upon it. Le­­ga­­cy co­m­­pa­­nie­s, adapt to this, so their so­­­ftwa­­re a­r­­chi­­te­c­­tu­­re be­­­co­­­mes the co­­­re of the bu­­si­­ness, and they ope­­ra­­te ba­s­ed on ­­this.

The rest of the ta­lks we­re ama­zing as we­ll, and in the ma­jo­ri­ty I cho­se ta­lks of the tra­cks of stream pro­ce­s­sing and pi­pe­li­nes wi­th so­me ex­cep­tions on whi­ch I atten­ded a ta­lk about in­ter­nals that ex­plai­ned key me­tri­cs to mo­ni­tor in a Ka­fka ins­ta­lla­tio­n, how Ka­fka uses the JVM heap in­ter­nal­s, and why this (a­lon­g wi­th a lot of mo­re me­tri­cs, su­ch as the Ker­nel I/O pa­gi­na­tion ca­che) are ke­y ­me­tri­cs to mo­ni­to­r.

At the end of the da­y, I had the chan­ce to ta­lk to so­me of the speakers an­d keep up en­li­gh­ten­ing dis­cus­sions about event sour­cing ar­chi­tec­tu­res.

Day 2

Shor­ter than the pre­vious one (fi­nis­hed at around 1pm), but the ta­lk we­re e­qua­lly ama­zin­g.

It was im­pres­si­ve to see how so­me big com­pa­nies use Ka­fka di­rec­tly on Do­cke­r ­con­tai­ner­s, wi­th Ku­ber­ne­tes, and a mo­ni­to­ring pla­tform au­to­ma­ti­ca­lly sen­din­g ­me­tri­cs to Gra­fa­na. The­re was a ta­lk ex­plai­ning the im­men­se in­fras­truc­tu­re at ­CER­N, and how Ka­fka pla­ys a cru­cial ro­le on their da­ta pro­ce­s­sing so­lu­tion­s.

All in all it was a great con­fe­ren­ce, on whi­ch I learnt a lot about Ka­fka, ­dis­tri­buted sys­te­ms, event sour­cing ar­chi­tec­tu­re. Be­si­des lear­ning about the ­te­ch­no­lo­gy it was great to see how com­pa­nies do rea­l-world im­ple­men­ta­tion of ­the­se so­lu­tion­s, and ex­chan­ge opi­nions wi­th fe­llow pro­fe­s­sio­nal­s.