Smoke testing openshift with ansible-galaxy

The ansible-galaxy ioggstream.ocp_health role can run a smoke test on openshift in minutes:

– etcd consistency
– rhn subscriptions
– master status
– registry, ipfailover and router instances

NOTE: it’s not a replacement of oadm diagnostics ;)


ansible-galaxy install ioggstream.ocp_health
# eventually tweak parameters
# vi /root/.ansible/roles/ioggstream.ocp_health/tests/ocp_health.yml
ansible-playbook --check /root/.ansible/roles/ioggstream.ocp_health/tests/ocp_health.yml

If you want to create a test project with two apps, one with a PVC and one with an ephemeral, set create_test_project.


ansible-playbook -v -e create_test_project=yes /root/.ansible/roles/ioggstream.ocp_health/tests/ocp_health.yml

Trace http calls with python-requests

Today python-requests is the de-facto standard library for rest calls.

As everything goes on TLS, you can trace api calls with the following:


import httplib as http_client
http_client.HTTPConnection.debuglevel = 1
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

MySQL JSON fields on the ground!

Having to add a series of custom fields to a quite relational application, I decided to try the new JSON fields.

As of now you can:

– create json fields
– manipulate them with json_extract, json_unquote
– create generated fields from json entries

You can not:

– index json fields directly, create a generated field and index it
– retain the original json datatype (eg. string, int), as json_extract always returns strings.

Let’s start with a simple flask app:

# requirements.txt
mysql-connector-python
Flask-SQLAlchemy==2.0
SQLAlchemy>=1.1.3

Let’s create a simple flask app connected to a db.

import flask
import flask_sqlalchemy
from sqlalchemy.dialects.mysql import JSON

# A simple flask app connected to a db
app = flask.Flask('app')
app.config['SQLALCHEMY_DATABASE_URI']='mysql+mysqlconnector://root:secret@localhost:3306/test'
db = flask_sqlalchemy.SQLAlchemy(app)

Add a class to the playground and create it on the db. We need sqlalchemy>=1.1 to support the JSON type!

# The model
class MyJson(db.Model):
    name = db.Column(db.String(16), primary_key=True)
    json = db.Column(JSON, nullable=True)

    def __init__(self, name, json=None):
        self.name = name
        self.json = json

# Create table
db.create_all()

Thanks to flask-sqlalchemy we can just db.session ;)

# Add an entry
entry = MyJson('jon', {'do': 'it', 'now': 1})
db.session.add(entry)
db.session.commit()

We can now verify using a raw select that the entry is now serialized on db

# Get entry in Standard SQL
entries = db.engine.execute(db.select(columns=['*'], from_obj=MyJson)).fetchall()
(name, json_as_string), = first_entry  # unpack result (it's just one!)
assert isinstance(json_as_string, basestring) 

A raw select to extract json fields now:

entries = db.engine.execute(db.select(columns=[name, 'json_extract(json, "$.now")'], from_obj=MyJson)).fetchall()

(name, json_now), = first_entry  # unpack result (it's just one!)
assert isinstance(json_now, basestring) 
assert json_now != entry.json['now']  # '1' != 1 

FullText Indexing IPv6 addresses with MySQL 5.7

MySQL 5.7 supports generated fields. This is particularly useful for searching the string representation of numeric stored ip addresses:

CREATE TABLE catalog(
ip varbinary(16) not null,
hostname varchar(64) not null,
label varchar(64),
ip_ntoa varchar(64) generated always as (inet6_ntoa(ip)) STORED, -- generate and store fields with the address representation
fulltext key (hostname, ip_ntoa, label)
);

When inserting values

INSERT INTO catalog(ip,hostname,label) VALUES
(inet6_aton('127.0.0.1'), 'localhost', 'lo'),
(inet6_aton('192.168.0.1'), 'gimli', 'stage,ipv4'),
(inet6_aton('fdfe::5a55:caff:fefa:9089'), 'legolas', 'router,ipv6'),
(inet6_aton('fdfe::5a55:caff:fefa:9090'), 'boromir', 'router,ipv6')

you can search in OR mode with

SELECT hostname FROM catalog WHERE
  MATCH(ip_ntoa, hostname, label)
  AGAINST('9089 router');
-- returns every entry matching ANY needle
***1***
hostname: legolas
***2***
hostname: boromir

Or exact matches

SELECT hostname FROM catalog WHERE
  MATCH(ip_ntoa, hostname, label)
  AGAINST('+9089 +router' in boolean mode);
-- returns ONE entry matching ALL needles
***1***
hostname: legolas

Jboss EAP: dumping configurations as scripts!

Today I found this nice tool to create configuration scripts for Jboss EAP from existing configurations.

$  git clone https://github.com/tfonteyn/profilecloner.git
$  cd profilecloner
$  export JBOSS_HOME=/opt/jboss/
$  mvn install
$  ln -s profilecloner.jar target/profile*jar
$  ./profilecloner.sh -f save-script-here.cli  –controller=$HOST –username=admin –password=secret /profile=full-ha antani-new-profile-name
$  cat save-script-here.cli

Enjoy!

One-click remote deployment with jboss and eclipse via maven

Create a maven goal which deploy to a remote jboss eap 6.x instance is quite simple.

Just add the following to your pom.xml

    <plugins>
			<plugin>
				<groupId>org.jboss.as.plugins
				<artifactId>jboss-as-maven-plugin
				<version>7.6.Final
				<configuration>
					<username>admin
					<password>Admin#1234
					<hostname>192.168.0.10
					<port>19999
				</configuration>
			</plugin>
      </plugins>


Then create a maven run configuration in eclipse having:
– Base directory set to the variable ${project_path} (so that it applies to the current project)
– Goal: jboss-as:deploy

Reverse engineering included

With ipython you can write a function, like:

prompt [1]# def parse(line):
    ip, host = line.split()
    return "{host} IN PTR {ip}".format(host=host,ip=ip)

To edit our function, just use %edit and reference the line

prompt [2]# %edit 1

Once you modify the function, you cannot reference the newer code with edit, as

prompt [3]# %edit 2

just references “%edit 1” and not the newer code.

In this case we can simply recover the last code of our function with

prompt [4]# from inspect import getsourcelines
prompt [5]# getsourcelines(parse)
(['def parse(line):\n',
'    ip, host = line.split()[:2]\n',
'    return "{host} IN PTR {ip}".format(host=host,ip=ip)\n'],
1)

cx_oracle segfault by default

Playing with the twisted adbapi connecting to oracle. Stressing a bit the application (an sftp server authenticating on oracle) I found that when the connection pool is exhausted the application crashed.

The (simple) solution is to instantiate the pool with the threaded keyword.

dbpool = adbapi.ConnectionPool("cx_Oracle", 
  uri, 
  threaded=True  )

Twisted is an event-based framework: all the blocking calls should be done outside the listening thread. The a(synchronous)dbapi provide separate threads for db connections. Using cx_oracle in a non thread-safe way is a reasonable cause of segfault.

So checking cx_Oracle docs we found that thread safety is off by default to gain some performance.