Types of Descriptors

Re­sum­ing from where we left of­f, on the pre­vi­ous post, on which we took A first look at de­scrip­tors, it’s time to ex­plore their dif­fer­ent type­s and how they work in­ter­nal­ly.

In Python, almost everything is represented with a dictionary. Objects are dictionaries. Classes are objects, hence they also are contained into a dictionary. This is denoted by the __dict__ attribute that objects have.

There are two types of descriptors: data descriptors and non-data ones. If a descriptor implements both [1] __get__() and __set__(), it’s called a data descriptor; otherwise is a non-data descriptor.

Read more…

A first look at descriptors

De­scrip­tors are one of the most pow­er­ful fea­tures of Python. The rea­son why they’re so pow­er­ful is be­cause they en­able us to con­trol the core op­er­a­tions (get, set, delete) [1], of an at­tribute in a giv­en ob­jec­t, so that we can hook a par­tic­u­lar code, con­trolled by us, in or­der to mod­i­fy, change, or ex­tend the o­rig­i­nal op­er­a­tion.

Read more…

Upcoming talk at PyCon CZ 2017

The pro­pos­al of my talk ti­tled “Dis­cov­er­ing De­scrip­tors“, was ac­cept­ed for the next edi­tion of Py­Con CZ, in Prague, Czech Re­pub­lic. Al­though the sched­ule is not yet ready, this talk will most like­ly be on June 8th.

The first two days of the con­fer­ence (June 8th and June 9th), will be for talk­s, whilst the last one (Sat­ur­day, June 10), will be for work­shops (sprints).

This is an­oth­er in­ter­est­ing op­por­tu­ni­ty to share and learn from the Python ­com­mu­ni­ty, as it was last year when I pre­sent­ed at Eu­roPy­thon 2016. On this oc­ca­sion, I ­plan to post an en­try with the events, per each day of the con­fer­ence.

Running PostgreSQL in memory with docker

Introduction

In­ter­act­ing with a database, can be a reg­u­lar task of a de­vel­op­er, and for that we would like to en­sure that we are de­vel­op­ing and test­ing in sit­u­a­tions close ­to a re­al im­ple­men­ta­tion; there­fore, us­ing the same data­base as in­ pro­duc­tion can help de­tect­ing is­sues ear­ly.

How­ev­er, set­ting up an en­tire data­base serv­er for de­vel­op­men­t, can be cum­ber­some. Hope­ful­ly nowa­days, mod­ern op­er­at­ing sys­tems like Lin­ux have ­great tools and fea­tures that we could take ad­van­tage of.

In par­tic­u­lar, I would like to set­up a sim­ple data­base lo­cal­ly us­ing dock­er, and stor­ing the da­ta in mem­o­ry.

The idea is sim­ple: run a dock­er con­tain­er with the im­age for Post­gresSQL, us­ing a tmpfs [1] [2] as stor­age for the data­base (a ramfs could al­so be used).

Read more…

2016 in review

A re­view of the events that took place this year, re­lat­ed to soft­ware engi­neer­ing and ar­chi­tec­ture.

The conferences, talks, and events

These are some of the events I par­tic­i­pat­ed in:

  • (2 Apr 2016) PRO­TO­SEC 2016 [1]: A tech­ni­cal con­fer­ence about net­work­ing, pro­to­col­s, se­cu­ri­ty, and re­lat­ed top­ic­s.
  • (Ju­ly 2016) Eu­roPy­thon 2016: I at­tend­ed sev­er­al talk­s, train­ings, work­shop­s, and pre­sent­ed a talk.
  • (10 Sep 2016) Python Cór­do­ba meet­up [6]: in­ter­est­ing meet­up where I p­re­sent­ed my talk at Eu­roPy­thon and at­tend­ed the sprints af­ter­ward­s.
  • (14 Sep 2016) Ar­q­Conf Meet­up ¿Có­mo es tra­ba­jar dis­tribui­do glob­al­mente? ­Ca­so Open Stack [5]: Host­ed in the Red Hat’s of­fices in Buenos Aires, this talk was a very in­ter­est­ing one, on which I learned about the col­lab­o­ra­tive way of work­ing of the Open Stack team, around the globe.
  • (17 Nov 2016) Python Barcelona meet­up.
  • (9 Dec 2016) Barcelona Spark meet­up [2]
  • (22 Dec 2016) Python Barcelona De­cem­ber meet­up [7]: I pre­sent­ed my talk about clean code in Python (yes, again :-), and there were two oth­er in­ter­est­ing talk­s. Mine was the first one of the evening. The sec­ond talk, was about zc.build­out, an in­ter­est­ing tool for man­ag­ing pack­ages and en­vi­ron­ments in Python, which I first heard of at Py­conAr (Python Ar­genti­na group) 2012. The third and last talk, p­re­sent­ed a project named aio­cache [8]: a cache sys­tem im­ple­ment­ed in asyn­cio, that sup­ports mul­ti­ple back­-end­s. The talk walked us through the ­code, its ar­chi­tec­ture, and some use cas­es with demos.

Code wise

I start­ed with a small con­tri­bu­tion to the coala project [3], and dur­ing the Eu­roPy­thon sprints I con­trib­uted to aio­http with a pull re­quest.

There were two new re­leas­es of my vim con­fig­u­ra­tion, ver­sions v0.3 and v0.4 [4], which is great be­cause is the sin­gle tool I use ev­ery day, and it is nice to have sev­er­al im­prove­ments on it.

I al­so en­hanced my dot files, but that is still in be­ta.

On general

A mi­nor re­lease of Python3.5, but most im­por­tant­ly: Python3.6 was re­cent­ly re­leased!

There were two re­leas­es of Fe­do­ra, 24 and 25, and there were lots of im­prove­ments and changes in the Lin­ux Ker­nel.

Re­gard­ing in­for­ma­tion se­cu­ri­ty, there were some is­sues in the Lin­ux Ker­nel, al­so in ap­pli­ca­tions (such as dirty COW, etc.), and a mas­sive DDoS at­tack that stroke the In­ter­net on Oc­to­ber, due to IoT de­vices be­ing com­pro­mised.

What to look forward to for the new year

  • I still look for­ward for more con­tri­bu­tions to open source, in par­tic­u­lar with CPython.
  • More func­tion­al pro­gram­ming (S­cala/Haskell may­be), low-lev­el pro­gram­ming in­ C, and re­search in the field of in­fos­ec.
  • Keep on par­tic­i­pat­ing on tech­ni­cal con­fer­ences.

And a lot more.

[1] https://www.protosec.org/en/
[2] https://www.meetup.com/Spark-Barcelona/events/235763327/
[3] https://github.com/coala/coala
[4] https://github.com/rmariano/vim-config/releases
[5] https://www.meetup.com/es/ArquitecturaDeSoftware/events/233824352/
[6] https://www.meetup.com/es-ES/Cordoba-Python-Meetup/events/233741709/
[7] https://www.meetup.com/python-185/events/236196808/
[8] https://github.com/argaen/aiocache

Python Barcelona November meetup notes

Last Thurs­day No­vem­ber 17, 2016, there was a Python meet­up in Barcelona [1], with a set of in­ter­est­ing talk­s.

The first one was host­ed by two rep­re­sen­ta­tives of the gov­ern­ment of Catalun­ya, and Barcelona city, and they pre­sent­ed the tech­ni­cal chal­lenges they are ­fac­ing, and the new stack pro­posed for on­com­ing project­s. In this re­gard, it was in­ter­est­ing to see how they have lots of lega­cy ap­pli­ca­tions writ­ten in­ J2EE, with Ja­va frame­work­s, that are out­dat­ed, dif­fi­cult to main­tain, and they ­men­tioned the idea of mi­grat­ing them to new, more mod­ern tech­nolo­gies. In this sense, there are al­ready projects in pro­gress, and they chose Python + Djan­go ­for the mi­gra­tion and re-im­ple­men­ta­tion of the lega­cy sys­tem­s.

This was a very in­ter­est­ing in the sense that more than this was ac­tu­al­ly p­re­sent­ed. In par­tic­u­lar the idea of how the gov­ern­ment wants to ac­tu­al­ly own their sys­tem­s, and there­fore they are now choos­ing open source soft­ware. It ­goes be­yond than mere­ly us­ing Python and Djan­go: they are al­so mi­grat­ing the ­work­sta­tion ma­chines to open source soft­ware (Ubun­tu), and try­ing to earn more ­con­trac­tors that are star­tups rather than huge multi­na­tion­als with pro­pri­etary ­soft­ware. The idea is still the same: they want open data, trans­paren­cy, and to ac­tu­al­ly own their sys­tem­s.

These sort of ini­tia­tives are gain­ing more and more trac­tion in the Eu­ro­pean U­nion, as many cities are start­ing to shift to­wards an open gov­ern­men­t, with­ open data, and more trans­paren­cy. I look for­ward to see­ing more projects in­ Python with open source tech­nolo­gies, at their GitHub ac­count [2], in the ­forth­com­ing month­s.

The sec­ond talk was about Con­da, and short­ly there­after a brief in­tro­duc­tion, the rest of the pre­sen­ta­tion was ­most­ly a de­mo (live cod­ing! :-), on which we saw sev­er­al ex­am­ples of in­stalling ­pack­ages, ex­port­ing the en­vi­ron­men­t, cre­at­ing a new one from a tem­plate, etc. It was use­ful to see both the strengths and short­com­ings of the tool, what can and ­can­not be done with it, and how it can be use­ful for de­vel­op­ers that have to deal with sev­er­al sys­tem de­pen­den­cies.

The third and last talk, was about PLONE, a CMS pro­jec­t writ­ten in Python, and with its 15-year-old tenure, is the old­est Python pro­ject (thing that I did not know). It was in­ter­est­ing to learn about it­s high­-lev­el ar­chi­tec­ture, com­po­nents, and the way they work. Above that, I would ­like to high­light the re­cap that it was done about many un­der­ap­pre­ci­at­ed zope li­braries, that have been avail­able in Python for a long time, do­ing an amaz­ing job. It is now in my list, to re­view them.

All in al­l, it was a good mee­tup, and I liked very much the tech­ni­cal lev­el of the talk­s, and the fact that the top­ics were very di­verse, yet in­ter­est­ing. I would like to con­tin­ue at­tend­ing these mee­tup­s, and in the ­fu­ture I might even sub­mit a talk.

[1] https://www.meetup.com/es-ES/python-185/events/235058828/
[2] https://github.com/AjuntamentdeBarcelona

Oncoming events of Python and Architecture in September

The next Sat­ur­day 10 of Septem­ber, I will be pre­sent­ing my talk “Clean Code in Python”, as pre­sent­ed in the pre­vi­ous Eu­roPy­thon 2016, at a lo­cal Python event in Cór­doba, Ar­genti­na [1] [2].

The fol­low­ing week, I will be at­tend­ing a lo­cal ar­chi­tec­ture meet­up (ar­q­con­f), on which a talk [3] about Open­Stack will be host­ed, ex­plain­ing how they work in a dis­trib­uted fash­ion.

I look for­ward to en­joy­ing both events!

[1] https://www.meetup.com/Cordoba-Python-Meetup/events/233741709/
[2] https://medium.com/@toniabdala/juntada-python-c%C3%B3rdoba-septiembre-db4abe5c4858#.yo0x1mqe3
[3] https://www.meetup.com/ArquitecturaDeSoftware/events/233824352/

Returning data in PostgreSQL

In or­der to get the best per­for­mance from our data­base en­gine, most of the ­times, we have to take ad­van­tage of its ex­clu­sive fea­tures. This has the dis­ad­van­tage of get­ting far­ther away from stan­dard, AN­SI code, and in the case of RDBM en­gi­nes, this al­so mean­s, most of the ORMs will not work.

Be­sides that, it could some­times be worth ex­plor­ing what we can gain by adapt­ing to the en­gine.

This is some­thing learnt from the feed­back of my talk giv­en at Eu­roPy­thon 2016. Af­ter the talk, an at­tendee told me that he liked it, and asked me about the code por­trayed in one of the slides, the one about dec­o­ra­tors with the ex­am­ple of mov­ing some records from one table ­to an­oth­er. He asked me if I ran code like that one in pro­duc­tion, to which I an­swered no: the code is en­tire­ly fic­tion­al, and it was done for the mere pur­pos­es of the ex­am­ples, so I just need­ed an ex­cuse for a task that could be ­done in two SQL state­ments, so the ex­am­ple with the dec­o­ra­tor can be shown.

The code of the ex­am­ple fol­lows:

INSERT INTO archive_orders
SELECT * from orders
WHERE order_date < '2016-01-01';

DELETE from orders WHERE order_date < '2016-01-01';

Then I learnt that there is an­oth­er way of writ­ing that sort of state­ments, with the “re­turn­ing” sen­tence of Post­greSQL.

It would be re-writ­ten like:

WITH deleted as (
    DELETE FROM orders WHERE order_date < '2016-01-01'
    RETURNING *
)
INSERT INTO archive_orders select * from deleted;

Or… the oth­er way around:

WITH to_delete as(
    INSERT INTO archive_orders
    SELECT id, description, order_date FROM orders WHERE order_date < '2016-01-01'
    RETURNING id
)
DELETE FROM orders where id in (select id from to_delete);

The in­ter­est­ing point here, is that it en­tails a sin­gle com­mand, in­stead of t­wo. So this can be done with a sin­gle call to the database, sav­ing an ex­tra round trip.

The point here is that the delete state­ment of Post­greSQL (as well as the rest of the state­ments, IN­SERT, UP­DATE, for in­stance), can be spec­i­fied to re­turn the da­ta they af­fec­t, and this can be used in an in­ter­me­di­ate ta­ble to pipe it to an­oth­er state­men­t.

By de­fault, if you run the delete state­men­t, it should re­turn the num­ber of af­fect­ed rows, like:

delete from orders where id > 3;
DELETE 4

But, we can make it to re­turn the rows them­selves:

delete from orders where id > 3 returning \*;
id|   description    |     order_date
----+------------------+---------------------
4 | to be archived   | 2014-12-09 00:00:00
5 | First sale order | 2016-07-17 00:00:00
6 | First sale order | 2016-07-20 00:00:00
7 | First sale order | 2016-07-24 00:00:00
(4 rows)

DELETE 4

Or, spe­cif­ic col­umns if we se­lect them:

delete from orders where id > 3 returning id, description;
id |   description
----+------------------
4 | to be archived
5 | First sale order
6 | First sale order
7 | First sale order
(4 rows)

DELETE 4

So, we can use the “re­turn­ing” fea­ture of Post­greSQL, to do in a sin­gle com­mand what we usu­al­ly would do in two or more, and in some cas­es, it might be ­some­thing worth ex­plor­ing. It was great learn­ing things like this one, and get­ting tips as a re­sult from the feed­back of the talk (it does not change the mean­ing, and the ex­am­ple could re­main the same for the afore­men­tioned rea­son­s; it is just an ex­am­ple :-).