Using http proxies in openshift java projects

To use http proxies with java in openshift you should know:

– that tools like maven don’t honor http_proxy & co environment variables
– that each container image has its own build script (assemble) that does or does NOT take http_proxy into account.

Always check the image documentation if you need proxies:

- https://docs.openshift.com/online/using_images/s2i_images/java.html
- https://access.redhat.com/solutions/1758313
- https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/7.0/html/red_hat_jboss_enterprise_application_platform_for_openshift/configuring_eap_openshift_image#configuring_eap_env_vars

A general and flexible solution is:

– to provide a configuration/settings.xml in your project, eg.

github.com/ioggstream/java-project.git
- pom.xml
- src/
- configuration/settings.xml

– add in settings.xml

openshift interpolates every *PROXY* environment variable stripping stuff, so you may not always be able to do

 
  <proxies>                                                                                                                                                                                                       
   <proxy>
...
      <host>${env.HTTP_PROXY_HOST}</host>
...    

JBoss images support the following variables via the `assemble` script:

– HTTP*_PROXY_HOST
– HTTP*_PROXY_PORT

Another solution is to:

– get the assemble from the given image you’re using (different images, different assemble)
– customize it so that it uses environment variables to build a custom settings.xml to be used within the build
– add it to .s2i/bin/assemble

Here’s an example assemble supporting proxies https://github.com/ivanthelad/openshift-jee-sample/blob/jws/.sti/bin/assemble

Smoke testing openshift with ansible-galaxy

The ansible-galaxy ioggstream.ocp_health role can run a smoke test on openshift in minutes:

– etcd consistency
– rhn subscriptions
– master status
– registry, ipfailover and router instances

NOTE: it’s not a replacement of oadm diagnostics ;)


ansible-galaxy install ioggstream.ocp_health
# eventually tweak parameters
# vi /root/.ansible/roles/ioggstream.ocp_health/tests/ocp_health.yml
ansible-playbook --check /root/.ansible/roles/ioggstream.ocp_health/tests/ocp_health.yml

If you want to create a test project with two apps, one with a PVC and one with an ephemeral, set create_test_project.


ansible-playbook -v -e create_test_project=yes /root/.ansible/roles/ioggstream.ocp_health/tests/ocp_health.yml

Customizing openshift deployments configuration files

You may need to customize a configurationfile for eg. an openshift-router or the registry.
If the dc supports the TEMPLATE_FILE environment, you can do it in three steps, otherwise you should find
a hook to mount the file in an expected location.

First get the original configuration file and modify it as desired. In this example, we are increasing the maximum allowed connections.

 # oc rsh router-xxx cat /var/lib/haproxy/conf/haproxy-config.template > haproxy-config.template
 # vim haproxy-config.template  # modify as desired, eg.

--- /var/lib/haproxy/conf/haproxy-config.template       
+++ /var/lib/haproxy/conf/custom/haproxy-config.template       
@@ -7,6 +7,7 @@
 {{ $workingDir := .WorkingDir }}
 global
   # maxconn 4096
+  maxconn {{env "ROUTER_MAX_CONNECTIONS" "20000"}}
   daemon
 {{ with (env "ROUTER_SYSLOG_ADDRESS" "") }}
   log {{.}} local1 {{env "ROUTER_LOG_LEVEL" "warning"}}
@@ -39,6 +40,7 @@

 defaults
   # maxconn 4096
+  maxconn {{env "ROUTER_MAX_CONNECTIONS" "20000"}}
   # Add x-forwarded-for header.
 {{ if ne (env "ROUTER_SYSLOG_ADDRESS" "") ""}}
   option httplog

1- create a configmap from your new template file, eg.
2- reference the new file via the TEMPLATE_FILE environment if supported
3- use the volume feature to mount the configmap as a file

 
 # oc create configmap router-haproxy-34 --from-file=haproxy-config.template
 # oc set env dc/router TEMPLATE_FILE=/var/lib/haproxy/conf/custom/haproxy-config.template
 # oc volume dc/router --add --overwrite     \
      --name=config-volume     \
      --mount-path=/var/lib/haproxy/conf/custom     \
      --source='{"configMap": { "name": "router-haproxy-34"}}'

Now verify and rollout the new config.

 oc describe dc router
 oc rollout latest router

Brief OpenShift troubleshooting

If you have issues after an automagic openshift-on-openstack deployment:

1. Remember: every buildconfig created *before* the registry is not authorized to push the images

2. Remember: hawkular is a java application. Startup is slow. Just click there and wait for the startup

3. Ansible is your friend. To get container logs, just


ansible all -m shell -a 'ls /var/log/containers/CONTAINER_NAME*'

ansible all -m shell -a 'cat /var/log/containers/CONTAINER_NAME*' > CONTAINER_NAME.log

4. If a container don’t startup during the deployment, a broken image may have been downloaded

Jun 1 23:30:36 dev-7-infra-0 atomic-openshift-node: I0601 23:30:36.234103 32913 server.go:608] Event(api.ObjectReference{Kind:"Pod", Namespace:"default", Name:"router-1-deploy", UID:"033670a9-470e-11e7-878f-fa163eac2bf7", APIVersion:"v1", ResourceVersion:"936", FieldPath:""}): type: 'Warning' reason: 'FailedSync' Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {\"message\":\"invalid header field value \\\"oci runtime error: container_linux.go:247: starting container process caused \\\\\\\"exec: \\\\\\\\\\\\\\\"/pod\\\\\\\\\\\\\\\": stat /pod: no such file or directory\\\\\\\"\\\\n\\\"\"}"

Cleanup docker repo


docker ps -aq | xargs docker rm
docker rmi 90e9207f44f0 --force

5. Run oadm diagnostics on the master ;)

6. Check #oc get hostsubnet

OpenShift cockpit quickstart

Enabling openshift cockpit with the latest releases is quite simple, but requires using a local system account.

1- Install cockpit


yum install cockpit cockpit-kubernetes
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 9090 -j ACCEPT
systemctl enable cockpit.service --now

2- Create a custom user to be used for cockpit administration


useradd -m -k /home/cloud-user cockpit
passwd cockpit #

3- access cockpit via a tunnel from the management network using the user cockpit credentials.


ssh -D11111 cloud-user@bastion
firefox http://master-ip:9090

Adding docker images to openshift 3.1

Openshift 3.1 is based on Kubernetes and Docker, and provides a small set of images including jboss EAP 6.4.

You can add new images in two steps:

1- create an ImageStream, that’s a docker image + a set of labels
2- create a Template using that ImageStream

To create the ImageStream read carefully the following description.

# Create the ImageStream
oc create -f - <<EOF
apiVersion: v1
kind: ImageStream
metadata:
  name: wildfly9-openshift
  namespace: openshift        # Set this to "openshift" if you want to make this image globally visible
spec:
  dockerImageRepository: docker.io/openshift/wildfly-90-centos7:latest  # The original docker hub repo
  tags:
  - annotations:
      description: Wildfly 9.0 S2I images.
      iconClass: icon-jboss
      sampleRef: 9.0.x 
      supports: wildfly:9,javaee:7,java:8,
      tags: builder,javaee,java,jboss
      version: "1.0"
    name: "1.0"
status:
  dockerImageRepository: ""