Category: Software Development

Articles about software design and programming

Categories
Resources Software Development

How to make linux shell scripts wait for a command to finish before running another

now you know

I am writing this so when other people google how to do it, they have something to find to save them time.

For days I tried to figure out how to make sure a command finished before another was run. I couldn’t find any information anywhere. If you are like me you may be thinking ( or wondering if ) that the shell just zooms through the commands you put in a script file without waiting for each to finish. It seems like this because everything is rushing by so quickly you can’t read it.

For days I was running scripts to install and configure my servers and it kept hanging so bad I couldn’t even ping the server.

I was running the following for example

sudo apt-get update
sudo apt-get dist-upgrade -y
sudo apt-get reboot

And since my server instances were hard freezing right up, I figured it must be zooming right on through causing an error.

Well come to find out after much research and someone on twitter finally confirmed to me that the shell automatically waits for each command to finish before executing the next. The shell doesn’t automatically run all commands encountered at the same time.

Now I must figure out what is actually locking my server instances up.

now you know
Categories
Software Development Web Security

SSH secure shell links videos and resources

SSH resources

I had a nice article but somehow it got screwed to hell and back. I have no idea what I did. I will not rewrite it. This is now purely a list of resources. I really hate tinymce for this. You can’t just past text from the internet because it auto adds H4’s for some no brain reason. So you have to switch to text view to paste text, then you can switch back and add the link. So helpful.

Articles

SSH keys – basically documentation on the subject by arch linux.

Tutorials point article covering ssh-keygen

Understanding the SSH Encryption and Connection Process – a really decent article going into the details of how SSH actually works underneath for anyone interested. I highly suggest reading this as it eliminates some of the questions you may have.

ssh-keygen – Generate a New SSH Key

SSH command – article about SSH on

SSH manual.– a 1990’s looking manual LOL basically the SSH documentation from what I can see.

Really good DigitalOcean article/manual about SSH

How to manage multiple SSH key pairs

SSH Keys with Multiple GitHub Accounts 

Configuring openssh for passwordless login- a guide by ubuntu about how to setup openssh to allow logins by ssh without passwords.

Host Vs HostName – you know just to be confusing

Fix: Pseudo-terminal will not be allocated because stdin is not a terminal – because SSH is stuck in 1990 and you need to give passwords and answer yes EVERYWHERE This totally makes scripting automated server deploys EXTREMELY HARD.

Securely add a host (e.g. GitHub) to the SSH known_hosts file

Managing Your SSH known_hosts Using Git

SSH known_host file syntax specification and information.

Videos

The below video gives a little history of how SSH came about. It also covers how SSH works to send and receive data.

This video goes into more depth about SSH and how to use it.

Categories
Resources Software Development

What does a dollar sign followed by a square bracket $[…] mean in bash?

I saw something similar to this in some code in one of my books
var=$[ $var1 - $var2 ]

I wanted to know what it did and why it was used. I’m a perfectionist with OCD.

Turns out it is deprecated from the BASH language.
Originally $[] was used to do math in Bash scripts to do Math known as arithmetic expansion.

So the new way in BASH is to use the following syntax.

var=$(( $var1 - $var2))

Basically what this syntax does is it allows you to do math more easily. Without the above syntax you have to escape certain characters like >< With the above syntax you can basically do math without escaping plus youcan use post-increment $var++, post-decrement $var– , logical and &&, logical or || bitwise math etc. It really helps you out.

Further links, resources and information

More info on stackexchange Same as the link above

Another good source of info about the (()) syntax as used in if and while statements is found in the book Linux Command Line and Shell Scripting Bible.  starting on page 325 If you don’t own the book I highly suggest it. I’ve found one errata so far and that is what this post is about.

More links and resources to BASH scripting

More links and linux resources

Categories
Software Development Web Development

Linux bash scripting command substitution aka $(command)

Linux has this syntax that looks like so:
$(command)

This is called command substitution. This allows you to get information about the execution of the command instead of having it it directed to STDOUT aka the terminal screen as usual.

That is very useful actually because you can run a command and store the output in a variable and use it anywhere you want later.

A simple example you can easily play with:

DIR_LISTINGS=$(ls -al)
echo $DIR_LISTINGS

This is so simple you don’t even have to add it to a script you can run it straight in your terminal in any directory you user owns.

More information can be found in this excellent book Linux command line and shell scripting bible page 277

More info about command substitution.

Bash manual reference.

Categories
Resources Software Development

Debian Ubuntu Linux debconf resources and information

The debconf programmers tutorial – excellent tutorial on what debconf is and how to use it.

debconf documentation

Using debconf to configure a system – article about using debconf, gives a little more explanation of what it is.

Installing MySQL with debconf – good article

Categories
Software Development Uncategorized

Linux xclip command makes command line life easier

I discovered a new tool today while adding my ssh keys to Github, something called xclip.

The xclip command makes it easy to capture output to the clipboard so you can paste it to another location like into a browser or word file etc.

The github docs above have you use it to copy your RSA key and save it for pushing your repository so you don’t have to supply a user name and password.

xclip -selection clipboard < ~/.ssh/id_ed25519.pub

The above tells xclip to put the contents of id_ed25519.pub into the clipboard. You can then use ctrl + v to paste it anywhere you need it.

Here is more info and examples on how to use xclip

Link to some man page  type info about the options it accepts as arguments.

Another link to similar info about xclip it’s options etc.

Categories
Software Development Web Security

Forced password changes are not good security policy

For so long I have read and heard that a GREAT security feature is to force your users to change passwords every so many days/months. Linux even has a built in feature for this.

This is a really stupid idea for many, many reasons. #1 a password is a password is a password. If a hacker can guess one they can guess another. Simply forcing users to change passwords is a false feeling of security.

Another reason it is a piss poor idea is, users usually use another form of their password so they can remember it, which again solves nothing whatsoever.

Another reason this is a stupid idea is people will often forget their password.

Another reason this is a bad idea is because users usually either write their password down or store it in a regular note type program on their phone, more are using password saving software. I once used a password saver, it worked great it saved all my passwords… except for the password to unlock it. I quit using them after that. LOL

You can read more about the treacheries of forced password cycling here.

You will notice others say the same things and more here.

Even Microsoft realizes this is bad idea that should be left in the past.

Categories
Resources Software Development

Ansible resources links videos and information

Ansible is used to provision/setup servers for your app.
Why is Ansible better than shell scripting?

Categories
Resources Software Development

Mysql resources links and information

This page contains links and videos to information about MySQL database.

More on MySQL option files in the documentation here.

Categories
Software Development Web Development

How to install and configure Golang development environment on Ubuntu Linux

I am still working on this article as all information I’ve found about how to set PATH system wide in Ubuntu is totally wrong. I’ve yet to find a way to set Go in the PATH system wide on Ubuntu. The info in Golang Docs is even wrong for Ubuntu. This article will be updated when I discover the secret of Ubuntu PATH.

I wanted to know why Go documentation suggested saving PATH as it did, I get tired of not knowing why things are suggested. In this article I dig a little into setting PATH on Ubuntu and Linux in general.

First off go to the Golang website and download the latest version of Go. It doesn’t matter if it is in your Downloads folder, the following command unpacks it to the proper location.

Follow the instructions to unpack it for example

tar -C /usr/local -xzf go1.15.6.linux-amd64.tar.gz

but with your version number you downloaded.

Now here is where I explain some things. The next step where it talks about setting the PATH environmental variable let me explain some things.

Where it says the following in the docs:

Add /usr/local/go/bin to the PATH environment variable.
You can do this by adding the following line to your $HOME/.profile or /etc/profile (for a system-wide installation):

These are the locations Linux usually will get environmental variables from.  Here is a link to explain /etc/profile I wanted to know what it did. Here is another link with more details. As you can see /etc/profile is one of he locations where linux gets things such as Environmental variables for the entire system.

The other $HOME/.profile refers to the logged in users home directroy .profile file. This is another location Linux looks for user environmental information. This article explains more about users profile files.

You can set the PATH there with this as they show :

export PATH=$PATH:/usr/local/go/bin

You put that in one of those files and what Linux does is it is adding that value to the current value for PATH you can also add it to the PATH variable for your entire system (not on ubuntu), located at /etc/environment  environment is a file. Open it with vim and you will see a really long string PATH=”longlines”

You can add to it by putting this at the end

:/usr/local/go/bin

So you will end up with something like
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin"

The way above using /etc/environment apparently isn’t system wide  on Ubuntu because of the way it disables the root account and uses something else a file called /etc/sudoers to store the PATH variable for the root user. Do not edit that directly you can destroy your login and system. User a tool called visudo.

The other ways work fine. If you add the path only to your regular user account in .profile then go won’t be available to root if you somehow need it.

/etc/profile (for a system-wide installation) is a little better because all users will have access. Otherwise each user you create you will have to add the PATH info to their .profile file, it gets to be a pain.

WARNING :
Doing it this way makes golang only available to the logged in regular user, at least on Ubuntu. /etc/environment is supposed to set PATH system wide but it doesn’t   on ubuntu.

I just wanted to dig and see why the docs suggested what it did. I get tired of everything in tech just telling people to do things without any explanation or links to info.

Next up setting configuration values, especially GOPATH – not so necessary from what I’ve been told. But the link explains the GOPATH and how to set it. I would do this because you will see many examples refer to GOPATH and you need to know what it is.