Almost Wrong Way To Send Docker Containers logs To ELK

In this article, we’ll walk through setting up a Docker-based ELK (Elasticsearch, Logstash, and Kibana) stack to collect, view, and send Docker logs. services: elasticsearch: image: elasticsearch:7.17.24 environment: - discovery.type=single-node volumes: - ./elasticsearch_data/:/usr/share/elasticsearch/data mem_limit: "1g" redis-cache: image: redis:7.4.0 logstash-agent: image: logstash:7.17.24 volumes: - ./logstash-agent:/etc/logstash command: logstash -f /etc/logstash/logstash.conf depends_on: - elasticsearch ports: - 12201:12201/udp logstash-central: image: logstash:7.17.24 volumes: - ./logstash-central:/etc/logstash command: logstash -f /etc/logstash/logstash.conf depends_on: - elasticsearch kibana: image: kibana:7.17.24 ports: - 5601:5601 environment: - ELASTICSEARCH_HOSTS=http://elasticsearch:9200 depends_on: - elasticsearch ElasticSearch Just create a folder named elasticsearch_data for storing data....

October 16, 2024 · 3 min · Alex Popescu

Backup PostgreSQL with Docker in S3 Object Storage

How can you backup your PostgreSQL to a Hetzner S3 object storage ? Here is how it’s done. Creating S3 Object Storage First, let’s create our S3 Object Storage: Next, we should create our S3 access key and secret: Backup and restore Let’s create a database with sample data: -- Create the database CREATE DATABASE music_streaming_platform; \c music_streaming_platform; -- Create users table CREATE TABLE users ( user_id SERIAL PRIMARY KEY, username VARCHAR(50) UNIQUE NOT NULL, email VARCHAR(100) UNIQUE NOT NULL, date_joined DATE DEFAULT CURRENT_DATE, country VARCHAR(50) ); -- Create artists table CREATE TABLE artists ( artist_id SERIAL PRIMARY KEY, artist_name VARCHAR(100) UNIQUE NOT NULL, genre VARCHAR(50), country VARCHAR(50), active_since DATE ); -- Create albums table CREATE TABLE albums ( album_id SERIAL PRIMARY KEY, album_name VARCHAR(100) NOT NULL, release_date DATE, artist_id INT REFERENCES artists(artist_id) ON DELETE CASCADE, genre VARCHAR(50) ); -- Create songs table CREATE TABLE songs ( song_id SERIAL PRIMARY KEY, song_name VARCHAR(100) NOT NULL, duration TIME NOT NULL, album_id INT REFERENCES albums(album_id) ON DELETE CASCADE, plays INT DEFAULT 0, release_date DATE ); -- Create playlists table CREATE TABLE playlists ( playlist_id SERIAL PRIMARY KEY, playlist_name VARCHAR(100) NOT NULL, user_id INT REFERENCES users(user_id) ON DELETE CASCADE, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); -- Create playlist_songs table (many-to-many relationship between playlists and songs) CREATE TABLE playlist_songs ( playlist_id INT REFERENCES playlists(playlist_id) ON DELETE CASCADE, song_id INT REFERENCES songs(song_id) ON DELETE CASCADE, PRIMARY KEY (playlist_id, song_id) ); -- Insert data into users table INSERT INTO users (username, email, country) VALUES ('musiclover_101', 'lover101@example....

October 12, 2024 · 5 min · Alex Popescu

Good Starter Cloud Init Config

Do you want a good starter cloud init config? Here is an example: #cloud-config users: - name: user groups: users, admin sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/bash ssh_authorized_keys: - <update with public ssh key> chpasswd: list: | root:<secure-password-here> expire: False packages: - fail2ban - ufw package_update: true package_upgrade: true runcmd: - printf "[sshd]\nenabled = true\nbanaction = iptables-multiport" > /etc/fail2ban/jail.local - systemctl enable fail2ban - ufw default deny incoming - ufw default allow outgoing - ufw allow 2022/tcp - ufw enable - sed -i -e '/^\(#\|\)PermitRootLogin/s/^....

October 1, 2024 · 2 min · Alex Popescu

Running a CLI Before Your Main App In Docker

Do you ever want to run a CLI before your main app in Docker? Here is a complicated way to do just that. We need a folder named success_flag that will contain a flag for the main app to know when the cli has finished running. First, we delete any existing success flag that remains: base: build: . container_name: base volumes: - ./success_flag:/success_flag command: bash -c "rm -f /success_flag/cli_success" restart: "no" Second, let’s run the database layer, in this case Postgres (but it can be any database engine)....

September 19, 2024 · 2 min · Alex Popescu

Linux Login Update Notifier Script With Python

Introduction Do you ever want to get a notification with the number of updates required when logging into XFCE in debian/ubuntu like systems ? There is a simple python script that can do just that. It uses notify-send command and paired with the magic command that returns the update count, we get the next python script: #!/usr/bin/env python3 import os import subprocess # Run the apt-get command and grep the output command = 'apt-get --simulate upgrade | grep "upgraded....

August 29, 2024 · 2 min · Alex Popescu