Notes on the Kafka Summit London 2018

The­se are so­me no­tes and takeawa­ys on the re­cen­tly ce­le­bra­ted Ka­fka Su­m­mi­t 2018 in Lon­don.

The con­fe­ren­ce was or­ga­ni­zed in th­ree pa­ra­llel tra­cks for ses­sions that we­re ­co­ve­ring stream pro­ce­s­sin­g, pi­pe­li­nes, and in­ter­nal­s. To get a good ex­pe­rien­ce, I atten­ded ta­lks of the th­ree ty­pes, but wi­th a li­ttle pre­fe­ren­ce to­war­d­s in­ter­nals and strea­ms.

It was a two­-­day con­fe­ren­ce wi­th lo­ts of va­lua­ble te­ch­ni­cal con­ten­t, awe­so­me ­ta­lks, speaker­s, and a lot mo­re. He­re are the hi­gh­li­gh­ts.

Day 1

The Keynotes

The first ke­y­no­te, “The Dea­th and Re­bir­th of the Event Dri­ven Ar­chi­tec­tu­re”, was the per­fect way to start the con­fe­ren­ce. To­ta­lly a min­d-o­pen­ing ta­lk, to­ ­get the ri­ght idea of what is the sta­te of the art in ter­ms of dis­tri­bute­d s­ys­te­ms, and even­t-­dri­ven ar­chi­tec­tu­res in mo­dern so­ftwa­re en­gi­nee­rin­g.

So­me of the main idea­s, ph­ra­ses, and con­cep­ts of this ta­lk:

  • Not just the ar­chi­tec­tu­res, but al­so the bu­si­ness are even­t-o­rien­te­d. You can ­thi­nk a bu­si­ness, eve­ry bu­si­ness, as a lar­ge set of even­ts oc­cu­rring at all ­ti­me­s. The ar­chi­tec­tu­re then, is just re­flec­ting this.
  • Eve­ry bu­si­ness is now di­gi­ta­l. Star­t-ups crea­te their ar­chi­tec­tu­re, and buil­d ­their bu­si­ness upon it. Le­ga­cy com­pa­nie­s, adapt to this, so their so­ftwa­re ar­chi­tec­tu­re be­co­mes the co­re of the bu­si­ness, and they ope­ra­te ba­sed on ­this.

The rest of the ta­lks we­re ama­zing as we­ll, and in the ma­jo­ri­ty I cho­se ta­lks of the tra­cks of stream pro­ce­s­sing and pi­pe­li­nes wi­th so­me ex­cep­tions on whi­ch I atten­ded a ta­lk about in­ter­nals that ex­plai­ned key me­tri­cs to mo­ni­tor in a Ka­fka ins­ta­lla­tio­n, how Ka­fka uses the JVM heap in­ter­nal­s, and why this (a­lon­g wi­th a lot of mo­re me­tri­cs, su­ch as the Ker­nel I/O pa­gi­na­tion ca­che) are ke­y ­me­tri­cs to mo­ni­to­r.

At the end of the da­y, I had the chan­ce to ta­lk to so­me of the speakers an­d keep up en­li­gh­ten­ing dis­cus­sions about event sour­cing ar­chi­tec­tu­res.

Day 2

Shor­ter than the pre­vious one (fi­nis­hed at around 1pm), but the ta­lk we­re e­qua­lly ama­zin­g.

It was im­pres­si­ve to see how so­me big com­pa­nies use Ka­fka di­rec­tly on Do­cke­r ­con­tai­ner­s, wi­th Ku­ber­ne­tes, and a mo­ni­to­ring pla­tform au­to­ma­ti­ca­lly sen­din­g ­me­tri­cs to Gra­fa­na. The­re was a ta­lk ex­plai­ning the im­men­se in­fras­truc­tu­re at ­CER­N, and how Ka­fka pla­ys a cru­cial ro­le on their da­ta pro­ce­s­sing so­lu­tion­s.

All in all it was a great con­fe­ren­ce, on whi­ch I learnt a lot about Ka­fka, ­dis­tri­buted sys­te­ms, event sour­cing ar­chi­tec­tu­re. Be­si­des lear­ning about the ­te­ch­no­lo­gy it was great to see how com­pa­nies do rea­l-world im­ple­men­ta­tion of ­the­se so­lu­tion­s, and ex­chan­ge opi­nions wi­th fe­llow pro­fe­s­sio­nal­s.