Types of Descriptors

Re­sum­ing from where we left of­f, on the pre­vi­ous post, on which we took A first look at de­scrip­tors, it’s time to ex­plore their dif­fer­ent types and how they work in­ter­nal­ly.

In Python, almost everything is represented with a dictionary. Objects are dictionaries. Classes are objects, hence they also are contained into a dictionary. This is denoted by the __dict__ attribute that objects have.

There are two types of descriptors: data descriptors and non-data ones. If a descriptor implements both 1 __get__() and __set__(), it’s called a data descriptor; otherwise is a non-data descriptor.

Leer más…

A first look at descriptors

De­scrip­tors are one of the most pow­er­ful fea­tures of Python. The rea­son why they’re so pow­er­ful is be­cause they en­able us to con­trol the core op­er­a­tions (get, set, delete) 1, of an at­tribute in a giv­en ob­jec­t, so that we can hook a par­tic­u­lar code, con­trolled by us, in or­der to mod­i­fy, change, or ex­tend the orig­i­nal op­er­a­tion.

Leer más…

Upcoming talk at PyCon CZ 2017

The pro­pos­al of my talk ti­tled “Dis­cov­er­ing De­scrip­tors“, was ac­cept­ed for the next edi­tion of Py­Con CZ, in Prague, Czech Re­pub­lic. Al­though the sched­ule is not yet ready, this talk will most like­ly be on June 8th.

The first two days of the con­fer­ence (June 8th and June 9th), will be for talk­s, whilst the last one (Sat­ur­day, June 10), will be for work­shops (sprints).

This is an­oth­er in­ter­est­ing op­por­tu­ni­ty to share and learn from the Python com­mu­ni­ty, as it was last year when I pre­sent­ed at Eu­roPy­thon 2016. On this oc­ca­sion, I plan to post an en­try with the events, per each day of the con­fer­ence.

Running PostgreSQL in memory with docker

Introduction

In­ter­act­ing with a database, can be a reg­u­lar task of a de­vel­op­er, and for that we would like to en­sure that we are de­vel­op­ing and test­ing in sit­u­a­tions close to a re­al im­ple­men­ta­tion; there­fore, us­ing the same data­base as in pro­duc­tion can help de­tect­ing is­sues ear­ly.

However, setting up an entire database server for development, can be cumbersome. Hopefully nowadays, modern operating systems like Linux have great tools and features that we could take advantage of.

In par­tic­u­lar, I would like to set­up a sim­ple data­base lo­cal­ly us­ing dock­er, and stor­ing the da­ta in mem­o­ry.

The idea is simple: run a docker container with the image for PostgresSQL, using a tmpfs 1 2 as storage for the database (a ramfs could also be used).

Leer más…

2016 in review

A re­view of the events that took place this year, re­lat­ed to soft­ware en­gi­neer­ing and ar­chi­tec­ture.

The conferences, talks, and events

These are some of the events I par­tic­i­pat­ed in:

  • (2 Apr 2016) PRO­TO­SEC 2016 1: A tech­ni­cal con­fer­ence about net­work­ing, pro­to­col­s, se­cu­ri­ty, and re­lat­ed top­ic­s.

  • (Ju­­ly 2016) Eu­­roPy­­thon 2016: I at­­tend­ed sev­er­al talk­s, train­ings, work­shop­s, and pre­sen­t­ed a talk.

  • (10 Sep 2016) Python Cór­do­ba meet­up 6: in­ter­est­ing meet­up where I pre­sent­ed my talk at Eu­roPy­thon and at­tend­ed the sprints af­ter­ward­s.

  • (14 Sep 2016) Ar­q­Conf Meet­up ¿Có­mo es tra­ba­jar dis­tribui­do glob­al­mente? Ca­so Open Stack 5: Host­ed in the Red Hat’s of­fices in Buenos Aires, this talk was a very in­ter­est­ing one, on which I learned about the col­lab­o­ra­tive way of work­ing of the Open Stack team, around the globe.

  • (17 Nov 2016) Python Barcelona meet­up.

  • (9 Dec 2016) Barcelona Spark meet­up 2

  • (22 Dec 2016) Python Barcelona De­cem­ber meet­up 7: I pre­sent­ed my talk about clean code in Python (yes, again :-), and there were two oth­er in­ter­est­ing talk­s. Mine was the first one of the evening. The sec­ond talk, was about zc.build­out, an in­ter­est­ing tool for man­ag­ing pack­ages and en­vi­ron­ments in Python, which I first heard of at Py­conAr (Python Ar­genti­na group) 2012. The third and last talk, pre­sent­ed a project named aio­cache 8: a cache sys­tem im­ple­ment­ed in asyn­cio, that sup­ports mul­ti­ple back­-end­s. The talk walked us through the code, its ar­chi­tec­ture, and some use cas­es with demos.

Code wise

I started with a small contribution to the coala project 3, and during the EuroPython sprints I contributed to aiohttp with a pull request.

There were two new releases of my vim configuration, versions v0.3 and v0.4 4, which is great because is the single tool I use every day, and it is nice to have several improvements on it.

I al­so en­hanced my dot files, but that is still in be­ta.

On general

A minor release of Python3.5, but most importantly: Python3.6 was recently released!

There were two releases of Fedora, 24 and 25, and there were lots of improvements and changes in the Linux Kernel.

Regarding information security, there were some issues in the Linux Kernel, also in applications (such as dirty COW, etc.), and a massive DDoS attack that stroke the Internet on October, due to IoT devices being compromised.

What to look forward to for the new year

  • I still look for­ward for more con­tri­bu­­tions to open source, in par­tic­u­lar with CPython.

  • More func­tion­al pro­gram­ming (S­cala/Haskell may­be), low-lev­el pro­gram­ming in C, and re­search in the field of in­fos­ec.

  • Keep on par­tic­i­­pat­ing on tech­ni­­cal con­fer­­ences.

And a lot more.

1

http­s://www.pro­to­sec.org/en/

2

http­s://www.meet­up.­com/S­park-Barcelon­a/events/235763327/

3

http­s://github.­com/­coala/­coala

4

http­s://github.­com/r­mar­i­ano/vim-­con­fig/re­leas­es

5

http­s://www.meet­up.­com/es/Ar­qui­tec­turaDeS­oft­ware/events/233824352/

6

http­s://www.meet­up.­com/es-ES/­Cor­doba-Python-Mee­tup/events/233741709/

7

http­s://www.meet­up.­com/python-185/events/236196808/

8

http­s://github.­com/ar­gaen/aio­cache

Python Barcelona November meetup notes

Last Thurs­day No­vem­ber 17, 2016, there was a Python meet­up in Barcelona 1, with a set of in­ter­est­ing talk­s.

The first one was host­ed by two rep­re­sen­ta­tives of the gov­ern­ment of Catalun­ya, and Barcelona city, and they pre­sent­ed the tech­ni­cal chal­lenges they are fac­ing, and the new stack pro­posed for on­com­ing project­s. In this re­gard, it was in­ter­est­ing to see how they have lots of lega­cy ap­pli­ca­tions writ­ten in J2EE, with Ja­va frame­work­s, that are out­dat­ed, dif­fi­cult to main­tain, and they men­tioned the idea of mi­grat­ing them to new, more mod­ern tech­nolo­gies. In this sense, there are al­ready projects in pro­gress, and they chose Python + Djan­go for the mi­gra­tion and re-im­ple­men­ta­tion of the lega­cy sys­tem­s.

This was a very in­ter­est­ing in the sense that more than this was ac­tu­al­ly pre­sent­ed. In par­tic­u­lar the idea of how the gov­ern­ment wants to ac­tu­al­ly own their sys­tem­s, and there­fore they are now choos­ing open source soft­ware. It goes be­yond than mere­ly us­ing Python and Djan­go: they are al­so mi­grat­ing the work­sta­tion ma­chines to open source soft­ware (Ubun­tu), and try­ing to earn more con­trac­tors that are star­tups rather than huge multi­na­tion­als with pro­pri­etary soft­ware. The idea is still the same: they want open data, trans­paren­cy, and to ac­tu­al­ly own their sys­tem­s.

These sort of ini­tia­tives are gain­ing more and more trac­tion in the Eu­ro­pean Union, as many cities are start­ing to shift to­wards an open gov­ern­men­t, with open data, and more trans­paren­cy. I look for­ward to see­ing more projects in Python with open source tech­nolo­gies, at their GitHub ac­count 2, in the forth­com­ing month­s.

The sec­ond talk was about Con­da, and short­ly there­after a brief in­tro­duc­tion, the rest of the pre­sen­ta­tion was most­ly a de­mo (live cod­ing! :-), on which we saw sev­er­al ex­am­ples of in­stalling pack­ages, ex­port­ing the en­vi­ron­men­t, cre­at­ing a new one from a tem­plate, etc. It was use­ful to see both the strengths and short­com­ings of the tool, what can and can­not be done with it, and how it can be use­ful for de­vel­op­ers that have to deal with sev­er­al sys­tem de­pen­den­cies.

The third and last talk, was about PLONE, a CMS project writ­ten in Python, and with its 15-year-old tenure, is the old­est Python project (thing that I did not know). It was in­ter­est­ing to learn about its high­-lev­el ar­chi­tec­ture, com­po­nents, and the way they work. Above that, I would like to high­light the re­cap that it was done about many un­der­ap­pre­ci­at­ed zope li­braries, that have been avail­able in Python for a long time, do­ing an amaz­ing job. It is now in my list, to re­view them.

All in al­l, it was a good mee­tup, and I liked very much the tech­ni­cal lev­el of the talk­s, and the fact that the top­ics were very di­verse, yet in­ter­est­ing. I would like to con­tin­ue at­tend­ing these mee­tup­s, and in the fu­ture I might even sub­mit a talk.

1

http­s://www.meet­up.­com/es-ES/python-185/events/235058828/

2

http­s://github.­com/A­jun­ta­ment­de­Barcelona

Oncoming events of Python and Architecture in September

El pr­óxi­mo sá­ba­do 10 de Sep­tiem­bre, es­ta­ré pre­sen­tan­do mi char­la “Clean co­de in Py­tho­n”, de la úl­ti­ma Eu­ro­P­y­thon 2016, en un even­to de Py­thon lo­cal en Cór­do­ba, Ar­gen­ti­na 1 2.

La otra se­ma­na, es­ta­ré asis­tien­do a un mee­tup lo­cal de ar­qui­rec­tu­ra (ar­q­con­f), en don­de se pre­sen­ta­rá una char­la so­bre OpenS­ta­ck 3 y su for­ma de tra­ba­jo dis­tri­buí­da.

Es­pe­ro dis­fru­tar de am­bos even­to­s!

1

http­s://www.­mee­tu­p.­co­m/­Cor­do­ba-­P­y­tho­n-­Mee­tu­p/e­ven­ts/233741709/

2

http­s://­me­diu­m.­co­m/@­to­nia­b­da­la/­jun­ta­da-­p­y­tho­n-­c%­C3%­B3r­do­ba-sep­tiem­bre-­db4a­be­5­c4858#.­yo­0­x1­m­qe3

3

http­s://www.­mee­tu­p.­co­m/Ar­qui­tec­tu­ra­De­So­ftwa­re/e­ven­ts/233824352/

Returning data in PostgreSQL

In or­der to get the best per­for­mance from our data­base en­gine, most of the times, we have to take ad­van­tage of its ex­clu­sive fea­tures. This has the dis­ad­van­tage of get­ting far­ther away from stan­dard, AN­SI code, and in the case of RDBM en­gi­nes, this al­so mean­s, most of the ORMs will not work.

Be­sides that, it could some­times be worth ex­plor­ing what we can gain by adapt­ing to the en­gine.

This is some­thing learnt from the feed­back of my talk giv­en at Eu­roPy­thon 2016. Af­ter the talk, an at­tendee told me that he liked it, and asked me about the code por­trayed in one of the slides, the one about dec­o­ra­tors with the ex­am­ple of mov­ing some records from one ta­ble to an­oth­er. He asked me if I ran code like that one in pro­duc­tion, to which I an­swered no: the code is en­tire­ly fic­tion­al, and it was done for the mere pur­pos­es of the ex­am­ples, so I just need­ed an ex­cuse for a task that could be done in two SQL state­ments, so the ex­am­ple with the dec­o­ra­tor can be shown.

The code of the ex­am­ple fol­lows:

INSERT INTO archive_orders
SELECT * from orders
WHERE order_date < '2016-01-01';

DELETE from orders WHERE order_date < '2016-01-01';

Then I learnt that there is an­oth­er way of writ­ing that sort of state­ments, with the “re­turn­ing” sen­tence of Post­greSQL.

It would be re-writ­ten like:

WITH deleted as (
    DELETE FROM orders WHERE order_date < '2016-01-01'
    RETURNING *
)
INSERT INTO archive_orders select * from deleted;

Or… the oth­er way around:

WITH to_delete as(
    INSERT INTO archive_orders
    SELECT id, description, order_date FROM orders WHERE order_date < '2016-01-01'
    RETURNING id
)
DELETE FROM orders where id in (select id from to_delete);

The in­ter­est­ing point here, is that it en­tails a sin­gle com­mand, in­stead of two. So this can be done with a sin­gle call to the database, sav­ing an ex­tra round trip.

The point here is that the delete state­ment of Post­greSQL (as well as the rest of the state­ments, IN­SERT, UP­DATE, for in­stance), can be spec­i­fied to re­turn the da­ta they af­fec­t, and this can be used in an in­ter­me­di­ate ta­ble to pipe it to an­oth­er state­men­t.

By de­fault, if you run the delete state­men­t, it should re­turn the num­ber of af­fect­ed rows, like:

delete from orders where id > 3;
DELETE 4

But, we can make it to re­turn the rows them­selves:

delete from orders where id > 3 returning \*;
id|   description    |     order_date
----+------------------+---------------------
4 | to be archived   | 2014-12-09 00:00:00
5 | First sale order | 2016-07-17 00:00:00
6 | First sale order | 2016-07-20 00:00:00
7 | First sale order | 2016-07-24 00:00:00
(4 rows)

DELETE 4

Or, spe­cif­ic col­umns if we se­lect them:

delete from orders where id > 3 returning id, description;
id |   description
----+------------------
4 | to be archived
5 | First sale order
6 | First sale order
7 | First sale order
(4 rows)

DELETE 4

So, we can use the “re­turn­ing” fea­ture of Post­greSQL, to do in a sin­gle com­mand what we usu­al­ly would do in two or more, and in some cas­es, it might be some­thing worth ex­plor­ing. It was great learn­ing things like this one, and get­ting tips as a re­sult from the feed­back of the talk (it does not change the mean­ing, and the ex­am­ple could re­main the same for the afore­men­tioned rea­son­s; it is just an ex­am­ple :-).