FullText Indexing IPv6 addresses with MySQL 5.7

MySQL 5.7 supports generated fields. This is particularly useful for searching the string representation of numeric stored ip addresses:

CREATE TABLE catalog(
ip varbinary(16) not null,
hostname varchar(64) not null,
label varchar(64),
ip_ntoa varchar(64) generated always as (inet6_ntoa(ip)) STORED, -- generate and store fields with the address representation
fulltext key (hostname, ip_ntoa, label)
);

When inserting values

INSERT INTO catalog(ip,hostname,label) VALUES
(inet6_aton('127.0.0.1'), 'localhost', 'lo'),
(inet6_aton('192.168.0.1'), 'gimli', 'stage,ipv4'),
(inet6_aton('fdfe::5a55:caff:fefa:9089'), 'legolas', 'router,ipv6'),
(inet6_aton('fdfe::5a55:caff:fefa:9090'), 'boromir', 'router,ipv6')

you can search in OR mode with

SELECT hostname FROM catalog WHERE
  MATCH(ip_ntoa, hostname, label)
  AGAINST('9089 router');
-- returns every entry matching ANY needle
***1***
hostname: legolas
***2***
hostname: boromir

Or exact matches

SELECT hostname FROM catalog WHERE
  MATCH(ip_ntoa, hostname, label)
  AGAINST('+9089 +router' in boolean mode);
-- returns ONE entry matching ALL needles
***1***
hostname: legolas

Adding docker images to openshift 3.1

Openshift 3.1 is based on Kubernetes and Docker, and provides a small set of images including jboss EAP 6.4.

You can add new images in two steps:

1- create an ImageStream, that’s a docker image + a set of labels
2- create a Template using that ImageStream

To create the ImageStream read carefully the following description.

# Create the ImageStream
oc create -f - <<EOF
apiVersion: v1
kind: ImageStream
metadata:
  name: wildfly9-openshift
  namespace: openshift        # Set this to "openshift" if you want to make this image globally visible
spec:
  dockerImageRepository: docker.io/openshift/wildfly-90-centos7:latest  # The original docker hub repo
  tags:
  - annotations:
      description: Wildfly 9.0 S2I images.
      iconClass: icon-jboss
      sampleRef: 9.0.x 
      supports: wildfly:9,javaee:7,java:8,
      tags: builder,javaee,java,jboss
      version: "1.0"
    name: "1.0"
status:
  dockerImageRepository: ""


Roundcube: risolvere l’errore Net_LDAP2_RootDSE::construct() must be public

Per rosolvere il seguente errore in roundcube

PHP Fatal error: Access level to Net_LDAP2_RootDSE::__construct() must be public (as in class PEAR) in roundcubemail/vendor/pear-pear.php.net/Net_LDAP2/Net/LDAP2/RootDSE.php on line 238

Seguire i passi:

  • cd <roundcube-root-folder>
  • Installare composer.phar: curl -s https://getcomposer.org/installer | php
  • copiare il template composer.json-dist in composer.json
  • modificare il file composer.json, nella sezione “require” aggiungere la riga"pear-pear.php.net/net_ldap2": "~2.2.0",
  • lanciare il comando: php composer.phar update

Back from MongoDB Essentials Training

This week I joined MongoDB Essentials training in Roma.

Mongo is a fast document oriented database supporting consistency, replication for HA and sharding for scaling read OR writes.

Transactions are at document level, so no joins and isolation levels.

A nice training – covering many database design technologies and giving even a theoretical overview of performance and scalability issues.

Concepts like Working Set, Replication types and issues, Indexes side-effects, Sharding and Hashing were introduced both theoretically and pratically.

Being a class of 10+ people with mixed background (MS, Linux, Oracle) it was hard to squeeze all this theory and practice in 3 days. The instructor asked us the parts we’d like to cover more: we pick Schema Design, Replication and Sharding.
Besides, such a large class give us a lot discussion and networking opportunity: we even created a freenode chatroom!

People interested in the subject can drop me a line and have a look at this github repo

Enjoy! R.

docker multihost network: an epiphany of namespaces.

Playing with docker multihost network this week-end.

With multihost networking you can run communicating containers on different docker nodes.
The magic relies on:
– a shared kv store (Eg. consul) for ipaddresses;
– a netns for vxlan for communication with a bridge and no processes attached.

Every network created using the Overlay driver has its own network namespace.
And for every network (& its subnet combination), we create a linux bridge inside that dedicated namespace.
The host end of the veth pair is moved into this namespace and attached to the bridge (inside of that namespace).
Hence, if you look for the veth pair in the host namespace, you wont find any :-).

If you look for vxlan setup on the boot2docker distro you have to dig deep ;).
1- docker netns is stored in /var/run/docker/netns. To access it you need to

#ln -s /var/run/docker/netns /var/run;

2- Now you can look for the vxlan netns, which has the same id on every machine:

#ip netns ls | while read a; do
    ip netns exec $a ip l | grep vxlan -q && echo $a;done

The vxlan references the UDP port for communication (eg. dstport 46354).

87: vxlan1:  mtu 1500 qdisc noqueue master br0 state UNKNOWN mode DEFAULT group default
    link/ether da:69:8d:4d:b9:39 brd ff:ff:ff:ff:ff:ff promiscuity 1
    vxlan id 256 srcport 0 0 dstport 46354 proxy l2miss l3miss ageing 300
    bridge_slave

3- Every container with EXPOSEd ports has a veth paired with a veth in the vxlan netns;

4- the veth in vxlan netns are slaves of br0;

5- br0 has an ip, and is the default gw for containers.

My Fedora is getting fat…

Monitor your Fedora disk usage:

abrt-cli [list|rm]
journald --disk-usage
[sudo] yum clean all

Store vm|docker data outside /var

virsh pool-edit default
#/etc/sysconfig/docker
OPTIONS="-g /data/docker"

Enable virsh for `wheel` users (from goldmann.pl)


sudo tee -a /etc/polkit-1/rules.d/80-libvirt.rules << EOF
# Don't ask password if user is in the wheel group.
polkit.addRule(function(action, subject) {
  if (action.id == "org.libvirt.unix.manage" && subject.local && subject.active && subject.isInGroup("wheel")) {
      return polkit.Result.YES;
  }
});

EOF

# Set context.
chcon --reference=/etc/polkit-1/rules.d /etc/polkit-1/rules.d/80-libvirt.rules

EuroPython 2015: insoliti incontri.

La diretta da EP continua, con insoliti incontri:

Guido van Rossum, creatore di Python e BDFL, ha rilasciato alcune importanti dichiarazioni: preferisce di gran lunga Dracula a Frankenstain, e preserverà la compatibilità tra python 3 e python 4.
With Guido van Rossum

Armin Rigo sta lavorando ad una nuova versione di pypy che migliora il multi-threading usando una logica Software Transactional Memory: l’accesso concorrente alla memoria viene gestito con un transaction log (eg. come fosse un db) anziché utilizzare dei lock.

Anche le pause caffè sono proficue: uno dei promotori del Barcelona Dojo – che lo scorso anno aveva seguito il mio training – mi ha aiutato a configurare una piattaforma per l’analisi dei log basata su Logstash (parser) -> Elasticsearch (database) -> Kibana (app di visualizzazione).

Non sono mancati poi i training: ho seguito quelli su MongoDB + Flask Web Framework e sulla Data Visualization.

Il talk più interessante e didattico invece, è stato quello sull’implementazione di sistema di configuration management geo-distribuibile basato su Consul.

E la conferenza continua: oggi è l’ultimo giorno di talk, domani e dopodomani ci saranno gli sprint – sessioni di hacking promosse dai maintainer dei vari software.

EuroPython 2015: Par-Tec c’è!

Anche quest’anno, come da 4 anni a questa parte, siamo presenti ad Europython. Nella magnifica cornice di Bilbao più di mille programmatori, sistemisti, devops e appassionati si sono riversati al check-in! [foto].
EP15
La location è fantastica, grazie al Google Social Event abbiamo potuto anche sperimentare la terrazza all’ultimo piano!

Il nostro talk è iniziato subito dopo il keynote, e nonostante la community python sia più vicina a mongodb e postgres, è stato bello vedere le persone fotografare lo schermo con le slide e chiedere informazioni alla fine. Pochissimi sapevano poi dell’esistenza di driver pure-python per MySQL!

La discussione innescata da Cloud Wars – un bel confronto tra i container provider (Atomic, CoreOS, …) dove ovviamente vince Atomic 😛 – è stata bella: più di un’ora a parlare dei problemi legati alla containerizzazione dei db insieme a sviluppatori di Openstack e vari cloud provider.

Qui ogni momento è geek, e anche se non ci siete potete seguire la diretta su twitter e/o farmi domande sui talk e le slide che trovate sul programma!

Altri talk interessanti:
Misurare i goal dello sviluppo
Garbage collection
Plotting big data