Overview
Red Hat OpenShift is an open-source container application platform based on the Kubernetes container orchestrator for enterprise application development and deployment. In this blog, I will be showing procedure on how to forward logs from Openshift Container Platform to vRealize Log Insight Cloud (vRLIC)
Once the logs are flowing you can create Dashboard to visualize your open shift environment like below I have created a sample dashboard
Pre-requisties
- Cluster Logging Operator is installed for openshift-logging namespace following the procedure listed here
Procedure
This procedure is applicable for 4.x and it has been tested with Version 4.3
The following section includes steps for running vRealize Log Insight Cloud Fluentd plugin as a Daemon set
Step 1
Generate vRealize Log Insight Cloud API Key from here
Step 2
Update the fluent.conf file with given configuration
@id in_tail_container_logs
@type tail
path
pos_file /fluentd-containers.log.pos
tag raw.kubernetes.*
read_from_head true
@type multi_format
format json
time_key time
time_format %Y-%m-%dT%H:%M:%S.%NZ
format /^(?
# Detect exceptions in the log output and forward them as one log entry.
@id raw.kubernetes
@type detect_exceptions
remove_tag_prefix raw
message log
stream stream
multiline_flush_interval 5
max_bytes 500000
max_lines 1000
# Concatenate multi-line logs
@id filter_concat
@type concat
key message
multiline_end_regexp /\\n$/
separator \"\"
@type record_transformer
fluentdhost ${hostname}
environment openshift
log_type kubernetes
# Enriches records with Kubernetes metadata
@id filter_kubernetes_metadata
@type kubernetes_metadata
watch false
@type vmware_log_intelligence
endpoint_url https://data.mgmt.cloud.vmware.com/le-mans/v1/streams/ingestion-pipeline-stream
verify_ssl false
Content-Type application/json
Authorization Bearer
structure simple
chunk_limit_records 300
flush_interval 3s
retry_max_times 3
@type json
tag_key text
: Complete path to Kubernetes access log file. The default path is /var/log/containers/*.log
: generated in Step 1
: Ensure the path have write access
: generated in Step 1
: Ensure the path have write access
Step 3
Create a configMap in Kubernetes for fluent.conf
kubectl --kubeconfig= create configmap --from-file=fluent.conf -n openshift-loggingStep 4
Create Daemon set vRLIC Fluentd YAML with following configuration
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentd-lint-logging
namespace: kube-system
labels:
k8s-app: fluentd-lint-logging
app: fluentd-lint-logging
version: v1
kubernetes.io/cluster-service: \"true\"
spec:
selector:
matchLabels:
name: fluentd-lint-logging
template:
metadata:
labels:
name: fluentd-lint-logging
app: fluentd-lint-logging
version: v1
kubernetes.io/cluster-service: \"true\"
spec:
serviceAccount: logcollector
serviceAccountName: logcollector
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
containers:
- name: fluentd-lint
image: docker.io/vmware/log-intelligence-fluentd
command: [\"fluentd\"]
env:
- name: FLUENTD_ARGS
value: --no-supervisor -q
resources:
limits:
memory: 500Mi
requests:
cpu: 100m
memory: 200Mi
securityContext:
privileged: true
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlogcontainers
mountPath: /var/log/containers
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: config-volume
mountPath: /etc/fluent
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlogcontainers
hostPath:
path: /var/log/containers
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: config-volume
configMap:
name: lint-fluent-config
- name: lint-fluent-volume
emptyDir: {}
- name: var-logs
emptyDir: {}
: Yaml uses logcollector service account which gets created when the cluster-logging operator is installed. If you plan to use any other service account then ensure it has rights to mount host path of /var/log/containers/*.log
Step 5
Apply the creation/changes of new daemon set configuration to the cluster.
kubectl --kubeconfig= apply -f lint-fluent.yml -n openshift-loggingIf everything is successful you can search for logs using filter environment contains openshift
Getting Started with vRealize LogInsight Cloud
For a free trial, you can click here or reach out to your account team
To learn more about vRealize Log Insight Cloud please visit here