💻
IT Documentation
  • 🥳Welcome!
  • General Concepts
    • SCRUM
      • Workflow
    • DevOps
      • What is DevOps?
      • What is TDD? (Test Driven Development)
      • What is CI? (Continuous Integration)
      • What is code coverage?
      • Linting best practices
      • Ephemeral Environments
      • Rolling Deployments
      • Blue/Geen Deployments
      • Canary Deployments
      • What is autoscaling & serverless?
      • What is service discovery?
      • What is Log Aggregation?
      • Metric Monitoring
  • AI
    • ChatGPT
      • Better prompts for ChatGPT
  • Cloud
    • AWS
      • AWS - 40 most common Services
      • AWS CLI Cheatsheet
      • Account & IAM
        • Create AWS Account
      • Lambda
        • Create Lambda function for Lightsail
      • Route 53
        • Set up R53 DNS Entry for GitBook
      • RDS
        • AWS RDS Burst Balance
      • VPC
        • What is a VPC
      • EC2
        • Removing EC2 user from sudo list
        • Create a Windows Gaming VPS
        • Connecting to a AWS EC2 instance
    • Azure
      • Courses
        • AZ900 Course
          • Overview
            • Describe Core Concepts
            • Cloud Models (IaaS, PaaS, SaaS)
          • Benefits of Cloud Computing
          • Cloud Models and Cloud Types
            • Overview of Cloud Models
            • IaaS, PaaS, SaaS
            • The Serverless Model
            • Cloud Types
          • Core Azure Architectural Components
            • Overview
            • Regions & Availability Zones
            • Resource Groups & Subscriptions & Management Groups
            • Resources & Resource Manager
          • Core Resources available
            • Compute Resources
            • Networking Resources
            • Storage Resources
            • Database Services
            • Azure Marketplace
          • Azure Core Solutions
            • Overview
            • Internet of Things (IoT) Solutions
            • Big Data Solutions
            • AI Solutions
            • Azure Functions & Logic Apps and Event Grid
            • DevOps Solutions
          • Azure Management Tools
            • Overview
            • Managing Azure with ARM Templates
            • Azure Monitor & Azure Service Health
          • Azure Security Features
            • Overview
            • Azure Security Center
            • Azure Key Vault & Azure Sentinel
            • Azure Dedicated Hosts
          • Azure Network Security
            • Network Security Group (NGS), Firewall and DDoS Protection
          • Azure Identity Services
            • Overview
            • Benefits
            • Multi-Factor Authentication (MFA)
            • Authentication vs Authorization
          • Azure Governance Features
            • Authentification and RBAC
            • Resource Locks
            • Azure Policy
            • Azure Blueprints
            • Cloud Adoption Framework (CAF)
          • Compliance Features
            • Core tenets of Security, Privacy and Compliance
            • Privacy Statement and Online Service Terms (OST)
            • Trust Center
            • Azure Sovereign Regions
          • Manage Azure Costs
            • Overview
            • Best practices
            • Pricing calculator
            • Azure Cost Management
          • SLA (Service level Agreements)
      • Virtual Machines
        • VM
          • Create a VM in a VNET
          • Azure VM LVM corruption fix
        • VMSS
          • Add SSH Key to VMSS
  • Containerization
    • Docker
      • Docker Cheatsheet
      • Install docker on Debian
      • Docker misc stuff
    • Docker Swarm
      • Docker Swarm Cheatsheet
      • Set up docker swarm
      • Delete docker swarm
      • Mount and bind volumes
      • Deploy Portainer via docker swarm
    • Docker Compose
      • Jenkins via Docker Compose
      • PostgreSQL via Docker Compose
      • Wireguard via Docker Compose & HTTPS
    • Rancher
      • Rancher installation guide
  • Cluster Computing
    • Slurm
      • Job Manager is not responding
      • Create new user
    • OpsCenter
      • Clear old Snapshots
      • Issues listing snapshots with nodetool
  • Database
    • General SQL
      • Database introduction
      • SQL 101
      • SQL Cheatsheet
      • User rights
      • Table Creation
      • SQL Replication - Best practice
      • SQL Database Design
    • MS SQL
      • Update whole table fast
    • Oracle SQL (PL/SQL)
      • Oracle SQL Cheatsheet
      • Oracle SQL - Kill Sessions
    • PostgreSQL
      • Install PostgreSQL
      • Create PostgreSQL Role and Database
      • Managing Postgres with PgAdmin GUI
      • Enable remote access for PostgreSQL
      • Authentication on PostgreSQL
      • Returning in Postgres
    • SQLite
    • Flyway Overview
    • SSRS Overview
    • Cassandra
      • GC OutOfMemoryError
  • DevOps
    • Ansible
      • Ansible Cheatsheet
      • Common Ansible Tasks
    • Git
      • Git 101
      • Git 1kb files
      • Git Commit changes before merge
      • Git Misc
      • Git Markdown
      • Git Clone Repo via SSH
    • Github
      • How to use multiple accounts
      • Delete commits fully
      • Set up git ignore file
    • Github Actions
      • Install self hosted runner
      • Scheduling jobs cron style
      • Passing ENV variable in script
      • SSH to Server
    • GitLab
      • GitLab Cheatsheet
    • Terraform
      • Terraform components
  • Hardware
    • UPS - Njoy
  • IoT
    • Home Assistant
      • Valetudo configs
      • Mini Media Player
      • HACS
    • Valetudo
      • Roborock Gen 1
      • Roborock quick guide
      • Roborock full Valetudo install guide
    • Tasmota
      • Tuya-Convert
    • LibreELEC
      • Quick LibreELEC guide
      • Configure X96 Mini Remote
  • OS
    • Linux
      • Learning guidelines
        • LPIC1 Notes
        • Linux Academy Notes
      • Install / Update Guides
        • Set up Raspberry PI
        • Update Debian 10 (buster) to Debian 11 (bullseye)
      • Increase disk size
      • umask
      • inodes
      • at jobs
      • yum
        • yum update vs yum upgrade
      • find
      • ssh
        • SSH returns: no matching host key type found. Their offer: ssh-rsa
        • Generate Public Key from Private Key
        • Run local bash scripts on remote server
      • crontab
        • Crontab 1st Sunday of every Month
        • Set crontab to execute after restart
      • vim
        • Use sed inside vim
      • networking
        • Check Port
      • fail2ban
      • bashrc
      • lvm
      • fallocate
        • Generate dummy file with actual size
      • openssl
        • Create Certificate via CNF file
        • OpenSSL cert conversion
    • Windows
      • Windows - Get App port by PID
      • Windows - Upgrade Windows build
      • Windows - Server
    • Android
      • Android - Motorola Unlock
      • Android - /E Project
    • PinePhone
      • PinePhone - Instructions for creating a PureOS image for PinePhone
  • Monitoring
    • Nagios
      • CPU threshold value calculation
    • New Relic
      • New Relic Flex Integration
      • NRQL Alerts examples
    • Zabbix
      • Zabbix Proxy not communicating with Windows Server
  • Microsoft Suite
    • Outlook
      • Change View
    • Excel
      • Excel Shortcuts
    • Windows Subset for Linux
      • WSL no internet connection
  • Networking
    • General Networking
      • IP Classes and Subnet Masks
      • Network CIDR Charts - /-es or IP Prefix
      • OSI Model Overview
      • Three Way Handshake & TCP Overview
    • F5
    • Authelia
      • What is Authelia
    • Nginx Proxy Manager
      • Nginx Proxy Manager - DuckDNS going down
    • Nmap
    • OpenWRT
      • Securing OpenWRT
      • OpenWRT - Read logs
      • OpenWRT - Adding DHCP Entry
      • OpenWRT - Wireguard
      • OpenWRT - Set up OpenVPN
      • OpenWRT - Internal DNS Service
      • OpenWRT - Set up new Wifi Interface
      • OpenWRT - Set up VLAN
      • OpenWRT - VPN Policy Routing
    • Pihole
      • Enabling HTTPS for your Pihole Web Interface
      • Edit Pihole DNS entries
    • RVS
      • RVS - Observer Modification
      • RVS - All Parameters
      • RVS - Adding a station
    • Wireguard
    • FTP
      • Connect to FTP anonymously
  • Pen Testing
    • CTF
      • CTF Links
  • Programming
    • Python
      • Classic Python
        • Python Cheatsheet
        • Python Shortcuts
        • Dunder Methods
        • hasattr(), getattr(), delattr()
        • Useful Exceptions
        • Dictionary
        • isinstance()
        • isdigit(), isdecimal(), isalpha()
        • return
        • Functions
        • Lists
        • ord(), chr()
        • squares, twos, odds
        • Bubble sort
        • append() and insert()
        • Bitwise operators
        • while, for & else
        • Arithmetic Operators
        • equal operators
        • Structure Projects
      • Modules
        • Webscraping
          • BeautifulSoup
        • PySimpleGui
          • Fast Crashcourse on PySimpleGui
        • os
        • python-docx
          • Generate DOCX file
        • psycopg2
          • PostgreSQL Connection
        • Pydantic Model vs SQLAlchemy Model
      • Frameworks
        • FastAPI
          • FastAPI Quick overview
          • Installing FastAPI and Dependencies
          • Starting FastAPI
          • Path Operations
          • Creating HTTP Operation paths
          • Send Data via Body of HTTP Request
          • Schema Validation with Pydantic
          • CRUD Operations
          • Storing in Array
          • Retrieve one individual entry
          • Changing response Status Codes
          • Deleting entries
          • Updating entries
          • API Documentation
          • Setup App Database & connect to database
          • FastAPI Response Model via Pydantic
          • Hashing passwords via FastAPI
          • Getting user by ID
          • FastAPI Routers
          • Router Prefix
          • Router Tags
          • JWT Token Basics
          • Login Process
          • Creating Token with OAuth2
          • OAuth2 PasswordRequestForm
          • Verify user is Logged In
          • Protecting Routes
          • Fetching User in Protected Routes
        • SQLAlchemy
          • What is an ORM
          • SQLAlchemy setup
          • Adding CreatedAt Column
          • CRUD via SQLAlchemy
          • Efficient way of passing params in SQLAlchemy
          • Creating Users Table via SQLAlchemy & FastAPI
      • Virtual Environments (venv)
    • General Programming Concepts
    • Interview Questions & Answers
      • General Programming Questions
      • Python Interview Questions Beginner
    • Courses
      • Python - PCAP-31-03 Course
        • Overview & Introduction
          • Exam Syllabus
          • Basics of variables
          • Basic Data Types
          • Basic Arithmetic in Python
          • Indexing and Slicing Strings
          • Basic String Methods
          • Format Method
          • Strings are Immutable
        • Lists, Tuples and Dictionaries
          • Lists
          • Accessing Elements in Nested Lists
          • Finding Index positions in Lists and counting duplicates
          • Tuples
          • Dictionaries
          • Comparison Operators
        • Functions and Variable Scope
          • Creating functions
          • *args and **kwargs
          • Basic Variable scope
          • Scope and Nested functions
        • Control Flow
          • If & Else Statements
          • Elif Statements
          • For Loops
          • Pass Statement in For Loops
          • While Loops
          • Looping and Unpacking with Dictionaries and Tuples
          • Range, Enumerate and Zip Functions
          • More Handy Functions and the Random Package
          • Accepting Input from User
        • Modules, Packages and OOP
          • Revising the Difference between Methods and Functions
          • Classes and Objects
          • Classes Attributes vs Object Attributes
          • Calling Python Code that is Saved in Another File
          • Inheritance and Polymorphism
          • Abstract Classes and Methods
          • Practical Application of OOP
          • Double Under (Dunder) Methods
          • Python Script Files
          • Python Files
          • Understanding the if __name__ == '__main__' Syntax
        • File IO and Exception Handling
          • Exception Handling
          • File IO
          • File IO with Exception Handling
          • OS Module
          • argv Command Line Arguments and the re Module
        • Misc Stuff and Q&A
    • IDE
      • Virtual Studio Code
        • Cheatsheet
    • Postman
      • Postman Overview
      • Create a GET HTTP request
      • HTTP Requests
      • Saving Postman requests
      • Environment Variables
  • Virtualization
    • Proxmox
      • Proxmox Cheatsheet
      • Proxmox Common Errors
      • Install Home Assistant in Proxmox via script
      • Create cloud-init template
      • Install guest-agent on new VM
      • Proxmox post install script
  • Webservers
    • Apache
      • Redirect 301 - Apache to index.html
    • Glassfish
      • Redirect 301 Glassfish
    • Tomcat
      • Useful tomcat files
  • Storage
    • NetApp
      • Netapp Overview
      • How to create symlinks
    • Nextcloud
      • Nextcloud Snap install and S3 Storage Bucket
      • Nextcloud Fail2Ban Regex
      • Set up OnlyOffice on Nextcloud
      • Set up Joplin and CalDav on Nextcloud
  • Software
    • Ansys
      • Ansys missing libraries
      • Ansys install
    • Jboss
      • Jboss process not working
Powered by GitBook
On this page
  • Shells
  • Commands
  • Confusion over Internal and External Commands
  • Shell Command Tricks
  • Search
  • Exploring Shell Configuration
  • Using Environment Variables
  • Getting Help (Manual Pages)
  • Using Streams, Redirection and Pipes
  • Exploring Types of Streams
  • Redirecting Input and Output
  • Commands for Streams, Redirects and Pipes
  • Piping Data Between Programs
  1. OS
  2. Linux
  3. Learning guidelines

LPIC1 Notes

Shells

  • CSH

  • TCSH

  • KSH

  • ZSH

  • BSH

  • BASH BASH is based on BSH and are the most popular.

BASH - based on BSH → Sometimes with symbolic link.

TCSH - based on csh

  • Not a default shell on any Distro

  • Similar to BASH

KSH - is designed to take the best features of BSH & CSH and improve/extend on them.

ZSH - takes evolution of shell even further then KSH, incorporating features from earlier Shells as well.

Symbolic Link to System’s default shell: /bin/sh

Commands

exit - terminates shell logout - terminates only login shell.

CTRL + A - move to start of line

CTRL + E - move to end of line

CTRL + B - Doubles as ←

CTRL + F - Doubles as →

CTRL + N - Doubles as arrow down

CTRL + P - Doubles as arrow up

Esc + B - Jumps back

Esc + F - Jumps forward

CTRL + C - Cancel

CTRL + L - Clear (or you can type “clear”)

BACKSPACE - Deletes Backwards DELETE - Deletes Forward

CTRL + K - Deletes everything from cursor forward.

CTRL + T - Takes characters before cursor and moves it forward.

Esc + T - Moves whole word forward Esc + U - Changes case (uppercase, lowercase) Esc + C - Converts letter under cursor to uppercase moving cursor to end of line.

CTRL (X+E) - Full editor from EMACS

Note: EMACS is a Editing system like VI (VIM)

cd - Change the working directory

pwd - Display the working directory (prints on the screen where you at!)

echo - displays the text you have entered (typing “echo Hello” causes the system to display the string “Hello”) - used for scripting

exec - runs an external program that you specified (executes program)

Note: The exec command special feature: Rather then creating a new process that runs alongside the shell, the new process replaces the shell, when the new process terminates, it is as if you terminated the shell.

time - shows how long subsequent commands take to execute. 3 Times are displayed:

  1. Total execution time (aka REAL TIME)

  2. User CPU time

  3. System CPU time

Example: time pwd - tells you how long the system took to execute the pwd command. Time displays after the full command terminates.

set - displays a wide variety of options relating to bash operations.

Note: These options are formatted much like environment variables but are not the same things.

exit / logout - both terminate the shell.

Note: exit - terminates any shell / logout - terminates only login shells (those are launched automatically when you initiate a text-mode login as opposed to those that run in xterm windows)

Confusion over Internal and External Commands

When duplicate internal and external commands exist, they sometimes produce subtly different results or accept different options.

These might create problems sometimes: Consider the pwd command and symbolic links to directories. Suppose you create a symbolic to /bin within your home directory and then cd into that directory. You then want to know where you are. The pwd command that’s internal to bash will produce a different result from the external pwd command.

$ pwd
/home/master/binlink
$ /bin/pwd
/usr/bin

As you can see, bash’s internal pwd show the path via the symbolic link, whereas the external command shows the path to which the link points.

The path is a list of directories in which commands can be found. It is defined by the PATH Environment Variable.

When you type a command that’s not recognized by the shell as one of its internal commands, the shell checks its path to find a program by that name and execute it.

WARNING: The root account should normally have a shorter path than ordinary user accounts, typically, you will omit directories that store GUI and other user-oriented programs from root’s path in order to discourage the use of the root account for routine operations, minimizing the risk of security.

In case of both programs on the path and paths you type as part of the command, the program must be masked as executable. - Done via the executable bit that is stored within the file.

In order to adjust a program’s executable status use chmod

Shell Command Tricks

Tab - Command completion; type part of command or filename; press tab; shell tries to fill in rest of command.

Note: Double-tap tab key for available options.

Note: Popular Linux Shells (bash, tcsh) support file completion, older shells might not.

History - Keeps record of every command you type.

Bash History is stored in the .bash_history file in your home directory.

history - brings up commands you typed (typically the last 500 lines)

history -c - clears history (which can be handy if you’ve recently typed commands you’d rather not have discovered by others, such as commands that include passwords)

Note: you can type the number of the command and execute it from history: !210 to execute command 210

Search

CTRL + R - Reverse Search → Type unique char for the command → Pressing CTRL + R searches through the commands containing the search.

Note: Once in the search you can hold CTRL + R for backwards search and CTRL + S for forwards search. CTRL + G terminates the search, or use arrow keys

If you want to edit the command after you have found it with search, you can move within the line with CTRL + A to move at start of the line or CTRL + E to move at the end of the line.

To delete text press: CTRL + D or the DELETE KEY which deletes the character under the cursor. Where by pressing BACKSPACE KEY deletes the character to the left of the cursor.

In order to launch a full-fledged editor to edit commands you can use CTRL + X followed by CTRL + E. - These commands are the most useful ones supported by bash.

Note: Bash attempts to launch the editor defined by the $FCEDIT or $EDITOR environment variable or Emacs as a last resort

Exploring Shell Configuration

Shells are configured through files that hold configuration options in a plain-text format.

The bash configuration files are actually bash shell scripts.

Main user configuration files for BASH:

  • ~/.bshrc

  • ~/.profile

Main global configuration files for BASH:

  • /etc/bash.bashrc

  • /etc/profile

Note: By editing these files you can change whatever needs changing, for instance you can add directories to the $PATH environment variable.

Be careful when making changes to configuration files, particularly the global configuration file. Best practice: Make backup of original, test changes, if problem occurs revert back to original config file.

Using Environment Variables

Environment variables are like variables in programming languages, they hold data to be referred to by the variable name.

You can add variables with the commands:

= export

Example:

ABC=google.com Export ABC

(This can also be done by using a single command)

export =

Example:

export ABC=google.com

There are programs that need this information and can refer to the environment variable:

Can be done from shell by using: echo $Variable

Example:

echo $ABC google.com

env - view the entire variable environment

unset - delete an environment variable

Example: unset ABC

Getting Help (Manual Pages)

man = manual; pages that provide summaries of what commands, features or files do.

Spacebar = move forward a page

Esc + V = move backward a page

/ = search for text

q = exit

info = info pages; are like man pages bu use hypertext format so you can move from section to section of the documentation for a program.

Using Streams, Redirection and Pipes

Streams, redirection and pipes are some of the more powerful command-line tools in Linux.

Linux treats input and output as streams, which is data that can be manipulated. You can manipulate the input or output ti come from/ go to other sources. Similarly you can pipe the output of one program into another.

Exploring Types of Streams

Standard Input = Programs accept keyboard input via standard input aka STDIN. In most cases this is data that comes into computer from a keyboard.

Standard Out = Text-mode programs send most data to their users via standard output aka STDOUT, which is displayed on the screen.

Standard Error = Linux provides a second type of output stream known as standard error aka STDERR. This output stream is intended to carry high-priority info such as error messages.

Note: Internally programs treat these streams just like data files. They open them, read from or write to the files, close them when they are done.

Redirecting Input and Output

To redirect output you use symbols following the command including options it takes.

Example:

echo $SHELL > shell.txt → result is that the shell.txt contains the output of the command in the @SHELL environment variable.

Commands for Streams, Redirects and Pipes

| = Creates a new file containing standard output. If file exists it will overwrite onto it.

|| = Appends standard output to the existing file. If file doesn’t exist, it creates it.

2> = Creates a new file containing standard error. If file exists, it overwrites it.

2>> = Appends standard error to the existing file. If the file doesn’t exist, it creates it.

&> = Creates new file containing both STDOUT and STDERR. If file exists, it overwrites it.

&>> = Appends STDOUT and STDERR to existing file. If file doesn’t exist, it creates is.

< = Sends the contents of the specific file to be used as standard input.

<< = Accepts text on the following lines as standard input.

<> = Causes the specific file to be used for both standard input and standard output.

| = Redirects STDOUT of one program to STDIN of a second program.

Note: The most important input redirection is < which takes the specified file content as standard input.

A common trick is to redirect STDOUT or STDERR to /dev/null - This file is a device that’s connected to nothing. It is used when you want to get rid of data.

tee = Command splits standard input so that it’s displayed on standard output and on as many files as you specify.

Piping Data Between Programs

Programs can frequently operate on other programs’ output. You can use one programs STDOUT as another program's’ STDIN

How it works:

Normally if you use redirection you will have to: Send first program’s STDOUT to a file → redirect 2nd program’s STDIN to read from that file.

By doing this it can lead to unnecessary cluttering on your system.

Solution is to use pipes (aka pipelines):

A pipe redirects the first program’s STDOUT to the second program’s STDIN.

Generating Command Lines:

xargs = command that builds a command from its standard input.

Example:

printf “1\n2\n3\n” = would print lines 1, 2, 3

printf “1\n2\n3\n” | xargs touch = would create file 1, 2, 3

printf “1\n2\n3\n” | xargs - i touch {}.txt = would replace text for 1 2, 3 with the extension text. (The brackets {} shows the xargs command where to replace the text)

PreviousLearning guidelinesNextLinux Academy Notes

Last updated 3 years ago

Good for research

Another xargs

link
Useful xargs video
intro