Types of Descriptors

Re­su­ming from whe­re we le­ft off, on the pre­vious pos­t, on whi­ch we took A first look at des­crip­tors, it’s ti­me to ex­plo­re their di­ffe­rent ty­pes and how they wo­rk in­ter­na­ll­y.

In Python, almost everything is represented with a dictionary. Objects are dictionaries. Classes are objects, hence they also are contained into a dictionary. This is denoted by the __dict__ attribute that objects have.

There are two types of descriptors: data descriptors and non-data ones. If a descriptor implements both [1] __get__() and __set__(), it’s called a data descriptor; otherwise is a non-data descriptor.

Leer más…

A first look at descriptors

Des­crip­tors are one of the most po­wer­ful fea­tu­res of Py­tho­n. The rea­son wh­y ­the­y’­re so po­wer­ful is be­cau­se they ena­ble us to con­trol the co­re ope­ra­tion­s (­ge­t, se­t, de­le­te) [1], of an attri­bu­te in a gi­ven ob­jec­t, so that we can hook a par­ti­cu­lar co­de, con­tro­lled by us, in or­der to mo­di­fy, chan­ge, or ex­tend the o­ri­gi­nal ope­ra­tio­n.

Leer más…

Upcoming talk at PyCon CZ 2017

The pro­po­sal of my ta­lk ti­tled “Dis­co­ve­ring Des­crip­tors“, was ac­cep­ted fo­r ­the next edi­tion of Py­Con CZ, in Pra­gue, Cze­ch ­Re­pu­bli­c. Al­thou­gh the sche­du­le is not yet read­y, this ta­lk wi­ll most like­ly be­ on Ju­ne 8th.

The first two da­ys of the con­fe­ren­ce (Ju­ne 8th and Ju­ne 9th), wi­ll be for ta­lks, whilst the last one (S­atur­da­y, Ju­ne 10), wi­ll be for wo­rks­hops (s­prin­ts).

This is ano­ther in­te­res­ting oppor­tu­ni­ty to sha­re and learn from the Py­tho­n ­co­m­mu­ni­ty, as it was last year when I pre­sen­te­d at Eu­ro­P­y­thon 2016. On this oc­ca­sio­n, I ­plan to post an en­try wi­th the even­ts, per ea­ch day of the con­fe­ren­ce.

Running PostgreSQL in memory with docker

Introduction

In­te­rac­ting wi­th a da­ta­ba­se, can be a re­gu­lar ta­sk of a de­ve­lo­pe­r, and for tha­t we would like to en­su­re that we are de­ve­lo­ping and tes­ting in si­tua­tions clo­se ­to a real im­ple­men­ta­tio­n; the­re­fo­re, using the sa­me da­ta­ba­se as in ­pro­duc­tion can help de­tec­ting is­sues ear­l­y.

Ho­we­ve­r, se­tting up an en­ti­re da­ta­ba­se ser­ver for de­ve­lo­p­men­t, can be­ ­cum­ber­so­me. Ho­pe­fu­lly no­wa­da­ys, mo­dern ope­ra­ting sys­te­ms like Li­nux ha­ve ­great tools and fea­tu­res that we could take ad­van­ta­ge of.

In par­ti­cu­la­r, I would like to se­tup a sim­ple da­ta­ba­se lo­ca­lly usin­g do­cker, and sto­ring the da­ta in me­mo­r­y.

The idea is sim­ple: run a do­cker con­tai­ner wi­th the ima­ge for Pos­tgresS­QL, u­sing a tm­pfs [1] [2] as sto­ra­ge for the da­ta­ba­se (a ra­m­fs could al­so­ ­be us­e­d).

Leer más…

2016 in review

A re­view of the even­ts that took pla­ce this yea­r, re­lated to so­ftwa­re en­gi­nee­ring and ar­chi­tec­tu­re.

The conferences, talks, and events

The­se are so­me of the even­ts I par­ti­ci­pated in:

  • (2 Apr 2016) PRO­TO­SEC 2016 [1]: A te­ch­ni­cal con­fe­ren­ce about ne­two­rkin­g, ­pro­to­col­s, se­cu­ri­ty, and re­lated to­pi­cs.
  • (Ju­ly 2016) Eu­ro­P­y­thon 2016: I atten­ded se­ve­ral ta­lks, tra­i­nings, wo­rks­hop­s, and pre­sen­ted a ta­lk.
  • (10 Sep 2016) Py­thon Cór­do­ba mee­tup [6]: in­te­res­ting mee­tup whe­re I ­pre­sen­ted my ta­lk at Eu­ro­P­y­thon and atten­ded the sprin­ts afte­rwar­d­s.
  • (14 Sep 2016) Ar­q­Conf Mee­tup ¿Có­mo es tra­ba­jar dis­tri­bui­do glo­bal­men­te? ­Ca­so Open Sta­ck [5]: Hos­ted in the Red Ha­t’s offi­ces in Bue­nos Ai­res, this ­ta­lk was a ve­ry in­te­res­ting one, on whi­ch I lear­ned about the co­lla­bo­ra­ti­ve way of wo­rking of the Open Sta­ck tea­m, around the glo­be.
  • (17 Nov 2016) Py­thon Bar­ce­lo­na mee­tup.
  • (9 Dec 2016) Bar­ce­lo­na Spa­rk mee­tup [2]
  • (22 Dec 2016) Py­thon Bar­ce­lo­na De­cem­ber mee­tup [7]: I pre­sen­ted my ta­lk a­bout clean co­de in Py­thon (ye­s, again :-), and the­re we­re two othe­r in­te­res­ting ta­lks. Mi­ne was the first one of the even­in­g. The se­cond ta­lk, was about zc.­buil­dout, an in­te­res­ting tool for ma­na­ging pa­cka­ges an­d en­vi­ron­men­ts in Py­tho­n, whi­ch I first heard of at Py­co­nAr (P­y­thon Ar­gen­ti­na grou­p) 2012. The third and last ta­lk, ­pre­sen­ted a pro­ject na­med aio­ca­che [8]: a ca­che sys­tem im­ple­men­ted in as­yn­cio, that su­ppor­ts mul­ti­ple ba­ck-en­d­s. The ta­lk wa­lked us th­rou­gh the ­co­de, its ar­chi­tec­tu­re, and so­me use ca­ses wi­th de­mo­s.

Code wise

I started wi­th a sma­ll con­tri­bu­tion to the coa­la pro­ject [3], and du­rin­g ­the Eu­ro­P­y­thon sprin­ts I con­tri­buted to aioh­ttp wi­th a pu­ll re­ques­t.

The­re we­re two new re­lea­ses of my vim con­fi­gu­ra­tion, ver­sions v0.3 an­d v0.4 [4], whi­ch is great be­cau­se is the sin­gle tool I use eve­ry da­y, an­d it is ni­ce to ha­ve se­ve­ral im­pro­ve­men­ts on it.

I al­so enhan­ced my dot fi­les, bu­t ­that is sti­ll in be­ta.

On general

A mi­nor re­lea­se of Py­tho­n3.5, but most im­por­tan­tl­y: Py­tho­n3.6 wa­s ­re­cen­tly re­lea­se­d!

The­re we­re two re­lea­ses of Fe­do­ra, 24 and 25, and the­re we­re lo­ts of im­pro­ve­men­ts and chan­ges in the Li­nux Ker­nel.

Re­gar­ding in­for­ma­tion se­cu­ri­ty, the­re we­re so­me is­sues in the Li­nux Ker­ne­l, al­so in appli­ca­tions (su­ch as dir­ty CO­W, etc.), and a ma­s­si­ve DDoS atta­ck tha­t s­tro­ke the In­ter­net on Oc­to­be­r, due to IoT de­vi­ces being com­pro­mi­s­e­d.

What to look forward to for the new year

  • I sti­ll look fo­rward for mo­re con­tri­bu­tions to open sour­ce, in par­ti­cu­la­r wi­th CP­y­tho­n.
  • Mo­re func­tio­nal pro­gra­m­ming (S­ca­la/Ha­ske­ll ma­y­be­), lo­w-­le­vel pro­gra­m­ming in ­C, and re­sear­ch in the field of in­fo­sec.
  • Keep on par­ti­ci­pa­ting on te­ch­ni­cal con­fe­ren­ce­s.

And a lot mo­re.

[1] https://www.protosec.org/en/
[2] https://www.meetup.com/Spark-Barcelona/events/235763327/
[3] https://github.com/coala/coala
[4] https://github.com/rmariano/vim-config/releases
[5] https://www.meetup.com/es/ArquitecturaDeSoftware/events/233824352/
[6] https://www.meetup.com/es-ES/Cordoba-Python-Meetup/events/233741709/
[7] https://www.meetup.com/python-185/events/236196808/
[8] https://github.com/argaen/aiocache

Python Barcelona November meetup notes

Last Thurs­day No­vem­ber 17, 2016, the­re was a Py­thon mee­tup in Bar­ce­lo­na [1], wi­th a set of in­te­res­ting ta­lks.

The first one was hos­ted by two re­pre­sen­ta­ti­ves of the go­vern­ment of Ca­ta­lun­ya, and Bar­ce­lo­na ci­ty, and they pre­sen­ted the te­ch­ni­cal cha­llen­ges they are ­fa­cin­g, and the new sta­ck pro­po­sed for on­co­ming pro­jec­ts. In this re­gar­d, it was in­te­res­ting to see how they ha­ve lo­ts of le­ga­cy appli­ca­tions wri­tten in ­J2EE, wi­th Ja­va fra­mewo­rks, that are ou­tdate­d, di­ffi­cult to main­tai­n, and the­y ­men­tio­ned the idea of mi­gra­ting them to new, mo­re mo­dern te­ch­no­lo­gie­s. In this sen­se, the­re are al­ready pro­jec­ts in pro­gress, and they cho­se Py­thon + Djan­go­ ­for the mi­gra­tion and re-im­ple­men­ta­tion of the le­ga­cy sys­te­ms.

This was a ve­ry in­te­res­ting in the sen­se that mo­re than this was ac­tua­ll­y ­pre­sen­te­d. In par­ti­cu­lar the idea of how the go­vern­ment wan­ts to ac­tua­lly own ­their sys­te­ms, and the­re­fo­re they are now choosing open sour­ce so­ftwa­re. It ­goes be­yond than me­re­ly using Py­thon and Djan­go: they are al­so mi­gra­ting the wo­rks­ta­tion ma­chi­nes to open sour­ce so­ftwa­re (U­bun­tu), and tr­ying to earn mo­re ­con­trac­tors that are star­tups ra­ther than hu­ge mul­ti­na­tio­nals wi­th pro­prie­ta­r­y ­so­ftwa­re. The idea is sti­ll the sa­me: they want open da­ta, trans­pa­ren­c­y, and to­ ac­tua­lly own their sys­te­ms.

The­se sort of ini­tia­ti­ves are gai­ning mo­re and mo­re trac­tion in the Eu­ro­pean U­nio­n, as many ci­ties are star­ting to shi­ft to­war­ds an open go­vern­men­t, wi­th o­pen da­ta, and mo­re trans­pa­ren­c­y. I look fo­rward to seeing mo­re pro­jec­ts in ­P­y­thon wi­th open sour­ce te­ch­no­lo­gie­s, at their Gi­tHub ac­count [2], in the ­for­th­co­ming mon­ths.

The se­cond ta­lk was about Con­da, and shor­tly the­rea­fter a brief in­tro­duc­tio­n, the rest of the pre­sen­ta­tion wa­s ­mos­tly a de­mo (li­ve co­din­g! :-), on whi­ch we saw se­ve­ral exam­ples of ins­ta­llin­g ­pa­cka­ges, ex­por­ting the en­vi­ron­men­t, crea­ting a new one from a tem­pla­te, etc. It wa­s u­se­ful to see bo­th the stren­gths and shor­tco­mings of the tool, what can an­d ­can­not be do­ne wi­th it, and how it can be use­ful for de­ve­lo­pers that ha­ve to­ ­deal wi­th se­ve­ral sys­tem de­pen­den­cie­s.

The third and last ta­lk, was about PLO­NE, a CMS pro­jec­t w­ri­tten in Py­tho­n, and wi­th its 15-­yea­r-old te­nu­re, is the ol­dest Py­tho­n ­pro­ject (thing that I did not kno­w). It was in­te­res­ting to learn about its hi­gh-­le­vel ar­chi­tec­tu­re, com­po­nen­ts, and the way they wo­rk. Abo­ve tha­t, I woul­d ­like to hi­gh­li­ght the re­cap that it was do­ne about many un­de­ra­ppre­ciated zo­pe li­bra­rie­s, that ha­ve been avai­la­ble in Py­thon for a long ti­me, doing an ama­zin­g ­jo­b. It is now in my lis­t, to re­view the­m.

All in all, it was a good mee­tu­p, and I liked ve­ry mu­ch the te­ch­ni­ca­l ­le­vel of the ta­lks, and the fact that the to­pi­cs we­re ve­ry di­ver­se, ye­t in­te­res­tin­g. I would like to con­ti­nue atten­ding the­se mee­tup­s, and in the ­fu­tu­re I mi­ght even su­bmit a ta­lk.

[1] https://www.meetup.com/es-ES/python-185/events/235058828/
[2] https://github.com/AjuntamentdeBarcelona

Oncoming events of Python and Architecture in September

El pr­óxi­mo sá­ba­do 10 de Sep­tiem­bre, es­ta­ré pre­sen­tan­do mi char­la “Clean co­de in Py­tho­n”, de ­la úl­ti­ma Eu­ro­P­y­thon 2016, en un even­to de Py­thon lo­cal en Cór­do­ba, Ar­gen­ti­na [1] [2].

La otra se­ma­na, es­ta­ré asis­tien­do a un mee­tup lo­cal de ar­qui­rec­tu­ra (ar­q­con­f), en don­de se pre­sen­ta­rá u­na char­la so­bre OpenS­ta­ck [3] y su for­ma de tra­ba­jo dis­tri­buí­da.

Es­pe­ro dis­fru­tar de am­bos even­to­s!

[1] https://www.meetup.com/Cordoba-Python-Meetup/events/233741709/
[2] https://medium.com/@toniabdala/juntada-python-c%C3%B3rdoba-septiembre-db4abe5c4858#.yo0x1mqe3
[3] https://www.meetup.com/ArquitecturaDeSoftware/events/233824352/

Returning data in PostgreSQL

In or­der to get the best per­for­man­ce from our da­ta­ba­se en­gi­ne, most of the ­ti­me­s, we ha­ve to take ad­van­ta­ge of its ex­clu­si­ve fea­tu­res. This has the ­di­sad­van­ta­ge of ge­tting far­ther away from stan­dar­d, AN­SI co­de, and in the ca­se of RDBM en­gi­nes, this al­so mean­s, most of the OR­Ms wi­ll not wo­rk.

Be­si­des tha­t, it could so­me­ti­mes be wor­th ex­plo­ring what we can gain by a­dap­ting to the en­gi­ne.

This is so­me­thing learnt from the fee­dba­ck of my ta­lk gi­ven at Eu­ro­P­y­thon 2016. After the ta­lk, an atten­dee told me ­that he liked it, and asked me about the co­de por­tra­yed in one of the sli­des, ­the one about de­co­ra­tors wi­th the exam­ple of mo­ving so­me re­cor­ds from one ta­ble ­to ano­the­r. He asked me if I ran co­de like that one in pro­duc­tio­n, to whi­ch I an­swe­red no: the co­de is en­ti­re­ly fic­tio­na­l, and it was do­ne for the me­re ­pur­po­ses of the exam­ple­s, so I just nee­ded an ex­cu­se for a ta­sk that could be­ ­do­ne in two SQL sta­te­men­ts, so the exam­ple wi­th the de­co­ra­tor can be sho­wn.

The co­de of the exam­ple fo­llo­ws:

INSERT INTO archive_orders
SELECT * from orders
WHERE order_date < '2016-01-01';

DELETE from orders WHERE order_date < '2016-01-01';

Then I learnt that the­re is ano­ther way of wri­ting that sort of sta­te­men­ts, wi­th the “re­tur­nin­g” sen­ten­ce of Pos­tgreS­Q­L.

It would be re-w­ri­tten like:

WITH deleted as (
    DELETE FROM orders WHERE order_date < '2016-01-01'
    RETURNING *
)
INSERT INTO archive_orders select * from deleted;

Or… the other way aroun­d:

WITH to_delete as(
    INSERT INTO archive_orders
    SELECT id, description, order_date FROM orders WHERE order_date < '2016-01-01'
    RETURNING id
)
DELETE FROM orders where id in (select id from to_delete);

The in­te­res­ting point he­re, is that it en­tails a sin­gle co­m­man­d, ins­tead of ­two. So this can be do­ne wi­th a sin­gle ca­ll to the da­ta­ba­se, saving an ex­tra ­round tri­p.

The point he­re is that the de­le­te sta­te­ment of Pos­tgreS­QL (as we­ll as the ­rest of the sta­te­men­ts, IN­SER­T, UP­DA­TE, for ins­tan­ce), can be spe­ci­fied to­ ­re­turn the da­ta they affec­t, and this can be us­ed in an in­ter­me­dia­te ta­ble to­ ­pi­pe it to ano­ther sta­te­men­t.

By de­faul­t, if you run the de­le­te sta­te­men­t, it should re­turn the num­ber of a­ffec­ted ro­ws, like:

delete from orders where id > 3;
DELETE 4

Bu­t, we can make it to re­turn the ro­ws the­msel­ve­s:

delete from orders where id > 3 returning \*;
id|   description    |     order_date
----+------------------+---------------------
4 | to be archived   | 2014-12-09 00:00:00
5 | First sale order | 2016-07-17 00:00:00
6 | First sale order | 2016-07-20 00:00:00
7 | First sale order | 2016-07-24 00:00:00
(4 rows)

DELETE 4

Or, spe­ci­fic co­lumns if we se­lect the­m:

delete from orders where id > 3 returning id, description;
id |   description
----+------------------
4 | to be archived
5 | First sale order
6 | First sale order
7 | First sale order
(4 rows)

DELETE 4

So, we can use the “re­tur­nin­g” fea­tu­re of Pos­tgreS­Q­L, to do in a sin­gle co­m­man­d what we usua­lly would do in two or mo­re, and in so­me ca­ses, it mi­ght be­ ­so­me­thing wor­th ex­plo­rin­g. It was great lear­ning things like this one, an­d ­ge­tting tips as a re­sult from the fee­dba­ck of the ta­lk (it does not chan­ge the ­mea­nin­g, and the exam­ple could re­main the sa­me for the afo­re­men­tio­ned rea­son­s; it is just an exam­ple :-).