Categories
Software Development Web Development Web Security

How to create ssh keys for admin user login without passwords

The idea is to have a way for an admins to SSH into a server without having to use passwords. This adds a level of security to your server setup. Without private keys you have to enter your user name and password. This can be less secure than generating SSH keys and adding your public key to SSH, plus with keys you don’t have to remember passwords.

First you need to generate the SSH keys. I prefer the ed25519 algorithm which is a newer one. You can get more info here.  

The code to create an ed25519 ssh key in the current users .ssh directory will look like this.


ssh-keygen -f ~/.ssh/key-name -t ed25519  

The -f flag tells ssh-keygen the name of the files you want to create. The above command would create key-name(private key) and key-name.pub(public) key, in the current users .ssh directory. The ~ is a Linux shortcut meaning /home/current_user/ so you don’t have to type all that.

The -t flag tells ssh-keygen what type of algorithm to use. If you don’t specify the -f flag and give the file a name, then both files are output in the current users .ssh directory as ed25519 and ed25519.pub

Categories
Software Development Web Development

Where does symfony php framework hide the errors?

I kept saying this over and over and looking everywhere for answers. I finally found this page through googling tons of terms and combos until I found it. I kept thinking my errors would be in the logs I set in my Nginx configs, or even in my PHP configs. But they were continually empty, I was going insane. I seriously blew up on twitter.

BLOWING UP

I thought Symfony was simply suppressing or not passing the errors to Nginx. So the logs are located at the place in the link above from documentation and not in the location you set in the Nginx configs or PHP configs.

By default, log entries are written to the var/log/dev.log file when you’re in the dev environment. In the prod environment, logs are written to var/log/prod.log

What made this confusing was, the docs don’t have a logical link flow when you are reading them trying to learn Symfony. You later find the SymfonyCasts which are better. But what makes it  most confusing is in the docs about configuring Nginx, it even shows the following.

    error_log /var/log/nginx/project_error.log;
    access_log /var/log/nginx/project_access.log;

To me this was showing how to set the error log. This does nothing by the way, not unless nginx itself has an error I guess.

For months I’ve wondered WTF, where are my error logs. I kept putting my app in dev mode so I could debug it via the browser.

Don’t do that!!!!!

To be honest getting Symfony working with Nginx is a pain in the ballsocks. The reason is, Nginx doesn’t pass environmental variables through to php scripts like Apache does/can. If you want that kind of fancy feature you must hack nginx up and use some perl script or something similar. Otherwise with Nginx you must set the environment variables twice, once in nginx and once in shell.

Why would you do that?

Why two locations? Yeah this really angered me and blew my mind at first too. As mentioned above Nginx doesn’t have any easy way to pass the environment variables you set at the Linux server level. This is important with Symfony because you often need to run things like Doctrine on the command line.

So I was setting my Nginx Environmental variables, the app would see them just fine. I’d go to run doctrine or tests and BOOM missing environmental variables like WTF? Or I would set them in the Linux environment, view them with printenv load the app in the browser and Nginx didn’t pass the values to my script. It took a lot of googling to figure that out with lots of trial and failure. To make matters worse, you have to change the environmental variable names in order to run tests so that symfony loads them, otherwise it hides the values.

WTF is happening?

I then found out through experimenting that you had to set the variables for the command line in the Linux environment too. How to permanently set Linux environmental variablees covers how to do that. It’s easier to just Bash script or ansible the entire process with Hashicorp packer than to try to manually maintain it all, setting vars in two different places etc.

So for months I’ve been going insane trying to find my error logs. Today I found the error logs.

Problem solved

 

Categories
Software Development Web Development

How to switch users in Linux Bash Shell script and execute multiple commands as different user

If you search you will find different answers to this. You can do this in multiple ways, here I will talk about 2 ways, single command and multiple commands.

First the idea is to switch from say root user to a named user you created or was created for you on your Linux server to run commands as not the root user. The reason you want to do this is so that everything isn’t owned by the root user. Or you are installing something like PHP Composer which barfs on you if you run it as root user.

You will see some saying to use su others saying to use sudo (some bs options etc.) You will also see really wrong answers on Stack. I have no idea why you would use sudo over su, you can google that. But I do know that su switches users. Here is an article goes into more detail of su vs sudo and when you use both.

Single command syntax

So the first way is to run a single command directly inline. If you are the root user you simply use su The syntax to do so is as follows:


su - username "commandToExecute [command options and arguments]"

It has been my experience that the Double ” Quotes are required or else the shell gets confused. You may be able to use single quotes if you don’t use any variables within the quotes.

Multiple commands syntax

To more easily issue multiple commands or long commands you need to use Linux heredoc syntax.
Heredoc uses <


su - $username <<SHT
     cd $serverDir
     php $composerFile install
SHT

Like I said you can use any Delimiter you want. It is tradition to use all caps for the word, it makes it easier to spot. The ending word (EOF here) has to have no spaces or words before it. You can list any number of commands within that syntax and all will be executed by the user.

NOTE: After the ending EOF the shell returns the user to whatever user you were/are logged in as before the lines of code. If you are logged in as root, you are returned to root. Also when you issue the su command you are moved out of the directory you are in. That is why I used cd to move back to the directory I needed to be in.

More links

More info about changing users on stack here.

Here is a link to heredoc syntax explanation and examples

More information and examples about heredoc in bash

Bash how write large amounts of text to a file

Categories
Software Development Web Development

What does prototypical Javascript look like?

Way back before modern times, like 10 years ago. Javascript had a much funkier way of defining objects. It was called prototypical inheritance. This is still how Javascript works, the classes, modules etc. were all recent additions to the language to make it easier to work with. It is not a very fun way to program because it is like looking at a GIANT JSON more than a class with methods.

So what did/does prototype inheritance look like? Well this…



function JsCollection() {
    this.jsObject = new Object();

}
JsCollection.prototype = {
    constructor: JsCollection,
    addNamedProperty: function (property, value) {
        // only add the property if it doesn't exist, return true if it was created
        //return false if it was not, to allow for testing before adding a new property
        var returnBool = false;
        if (!this.jsObject.hasOwnProperty(property)) {
            returnBool = true;
            this.jsObject[property] = value;
        }
        return returnBool;
    },
    getElementCount: function () {
        var elementCount = 0;
        //loop through the object and add to the count
        for (var elem in this.jsObject) {
            //only add to the value if it is part of collection
            //https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Working_with_Objects
            if(this.jsObject.hasOwnProperty(elem)){
                     elementCount++;
            }
           
        }
        return elementCount;
    },
    getElementValue: function (property) {
        var returnProp = null;

        if (this.jsObject.hasOwnProperty(property)) {

            returnProp = this.jsObject[property];
        }
        return returnProp;
    },
    removeNamedProperty: function (property) {

        if (this.jsObject.hasOwnProperty(property)) {
            delete this.jsObject[property];
        }
    },
    changePropertyValue: function (property, value) {

        if (this.jsObject.hasOwnProperty(property)) {
            this.jsObject[property] = value;
        }
    },
    getAllNamedProperties: function () {
        return this.jsObject;
    },
    namedPropertyExists: function (property) {
        var propExists = false;
        if (this.jsObject.hasOwnProperty(property)) {
            propExists = true;
        }
        return propExists;
    }
};

//var objProps = obj1.getAllNamedProperties();
// how to loop through object properties 
//for(var prop in objProps){
//    console.log("Property is " + prop + ' Property value is ' + objProps[prop]);
//}

As you can see this is an object with functions in old fashioned Javascript syntax. This was too confusing of a syntax for most people, plus you had to learn the inner workings of Javascript and how prototypical inheritance works.

I won’t try  to explain it here as it is pretty complicated to wrap your head around. Some videos may help better than an article.



Categories
Software Development Web Development

How to configure php 7.4+ on Linux Ubuntu 20.04+

Configuring php can be confusing. PHP uses multiple configuration files, but the main ones are both named php.ini. You see with php you can have separate configurations for the app and the command line aka cli.

Why would you do that?

This may sound stupid at first but it is due to the fact that your command line is a different environment from your app environment.  In Linux for example each user of the cli has their own environmental variables. These variables get passed to the php cli. This can drive you insane on linux. You think you set the configurations correctly, app works, try command line and BOOM all messed up.

But I set the damn configs WTF?

The two main files are located at

  1. /etc/php/7.4/cli/php.ini
  2. /etc/php/7.4/fpm/php.ini

Yes they have the same exact name. Yes they contain the same thing. However, one is used for the command line (cli) and the other is for apps (fpm). So you must set the settings in both. If you wanted to use 1 file for both you might be able to remove/rename 1 file such as the cli, set the fpm file. Then use a symlink from the fpm file to the cli directory. It would require some testing but it will probably work.

P.S. The php configuration file is the largest you will ever in your life see. I’m guessing a few thousand lines, it feels like that, but there is a ton of commenting and documentation to help you complete with links.

Categories
Resources Web Development

Linux su command not working, does nothing but show $ prompt

So you found the magic sauce did ya?  So you created a user with useradd or adduser and you try to switch over to that user in a terminal, probably logged in through ssh as root right.

You are trying to use the su – username command but all you get is a $. And not the good kind. The kind that no matter what you type all you get is another line with $ on it. This is a feature by the way so you can’t see files that don’t belong to the user…

Right now you are probably like

Wait. WTF is even happening?

As far as I can tell, if you are using Debian or Ubuntu, useradd/adduser defaults the users shell to /bin/sh but the skeleton files located in /etc/skel are all configured for bash.  I have no idea how the system gets the defaults, but it does no good to have your users default shell not pointing to bash.

How to fix this?

To fix it you need to change the users default shell to bash. Bash is usually located in /bin/bash or /usr/bin/bash For me it was /bin/bash. To change it you use usermod command like so


usermod -s /bin/bash username

That will change the shell your user gets when you type su – username. Now since Ubuntu/Debian and maybe other distros contain the configuration in the users .bashrc and .profile files everything will work as expected. When you switch to the user with the above command you are taken to their home directory.

Found more details!

More info

Ok I found more info while digging into the so called useradd docs.

-s–shell SHELLThe name of the user’s login shell. The default is to leave this field blank, which causes the system to select the default login shell specified by the SHELL variable in /etc/default/useradd, or an empty string by default.

So that is where Linux gets the default value for the user shell and apparently you can use the -s option when creating the user to specify the bash as the shell.

Digging deeper into the mystery sauce I find in my Mastering Ubuntu Server book ( awesome book ) the reason why. It states if you use adduser then their default shell is /bin/bash and if you use useradd (which I used) it defaults to /bin/sh

Links

how to change the default shell of an user in linux? – more info about usermod and changing the users default shell and how to figure out what shells you have installed and their locations.

Categories
Software Development Web Development Web Security

How to create ssh deploy keys for github

One issue with creating SSH keys is there are so many ways to do it and no one tells you why they do what they do. A quick search will reveal almost everyone has their own way of doing it.  If you are new to ssh keys I suggest you read this article really quick.

In the github docs they tell you how to create the ssh keys like this

ssh-keygen -t ed25519 -C "your_email@example.com" 

I prefer using the following command instead :

ssh-keygen -t ed25519 -f /home/akashicseer/tests/ssh/file_name -C "akashicseer@gmail.com"

Quick command facts:

  1. ed25519 is basically the newest type of key, it is supposed to be the most secure
  2. -C is for adding a comment to the key. This helps you identify it later in places like ~/.ssh/known_hosts, ~/.ssh/authorized_keys and when you use the command ssh-add -L which prints out your public key info
  3. -t specifies the type of key. The above command tells ssh-keygen to create an ed25519 type of key more info
  4. -f /home/akashicseer/tests/ssh/file_name This tells ssh-keygen where to put the file. If you don’t specify the name then it will use a default of something like id_ed25519 for private key and id_ed25519.pub for the public key. The code above will put the files named file_name (private key) and file_name.pub ( public key) in the folder /home/akashicseer/tests/ssh/ If you don’t specify the full path to the exact folder your keys will be put into your users home directory in the default .ssh location. On Ubuntu this is /home/username/.ssh/

NOTE: for ssh deploy keys, don’t specify a passphrase when you create them or you will have to manually enter it later when Packer or whatever you use runs your provisioning code. That means you won’t be able to automate if you enter a passphrase, because it will ask the terminal user to enter the phrase to do a git clone.

There are different types of ssh keys. If you don’t add the ed25519 part then a regular ssh key of type rsa is created, this is the default type of ssh key. Basically Github documentation is showing how to create a secure type of key to use with your code deployments. You will use this key to clone your repository to your server instances.

Creating the key is only half the battle. You must decide how you will create the key, especially if you are automating deployments. When automating deployments the process becomes very complicated.

First either you create the keys you need in the instance you are creating then use the github api to add them to the proper repo. Or you create the keys on a local development computer and use something like Hashicorp Packer to upload the files to the server instance during creation. The latter is the easiest way especially in automation of the infrastructure.

If you are creating your keys locally and using Packer to upload them, you will need to login to Github and go to the deploy keys section of the specific repo to add your public key. The public key is the one that ends in .pub usually. The easiest way to copy the key value is to use xclip which I mention in this article.

If fully automating the process and creating the key on the actual instance, you must remember to eventually remove older keys. Github lets you have like 50 keys per repo max. If your repo needs to be deployed to many instances, such as a microservice structure you can contact them to get added key abilities. You could also reuse the same key, but that would require keeping the private key in a repo as well which probably isn’t a very good idea at all, since ssh keys are the same as passwords basically.

Also remember this. If you are using deploy keys only to deploy by cloning the repo, then deleting the key after the clone is perfectly fine. You only need to use this key one time to clone (aka deploy ) your code, after this it is useless. You can and probably should create new SSH keys for deployment each time and remove them from Github after you deploy, then delete them from the server instance.

Unless you plan on keeping the same instance up and trying to pull from the repo etc. That is messy. Personally I’d prefer to use Packer to create new instances when I need to and redeploy. This has the added benefit that I can upgrade the instance with security etc., test it, add my app code, test it, then swap over after migrating the database and other files. This is like creating a clean slate every time.

You will also need to know how to add the keys to ssh-agent and use them, which I cover in this article.

Here is a link to a list of resources about ssh.

Categories
Resources Software Development Web Development Web Security

SSL links, videos and other resources.

SSL is a very important subject. All websites/apps should be using it. However the docs will leave you scratching your head saying WTF? So I am creating this long list of resources for anyone else who ever has to learn how to use it.

Articles

First here is a link to the docs – this will cause confusion as nothing tells you how to use the pieces together.  So it is like looking into a box of legos and knowing it builds something but you don’t even have a picture as a hint. The best you can do is use the pieces to build something that doesn’t even resemble the original creation.

OpenSSL quick reference by digicert – a very brief introduction to SSL and how it works

SSL Certificate Security Glossary – list of terms and definitions

How to create a CSR with openssl – shows some of the syntax for the -config file option.

Docs explaining the config file found in the article above bout how to create a csr with openssl

SSL Basics: What is a Certificate Signing Request (CSR)? – Exactly WTF is a CSR

Openssl config file example – openssl docs are pure 100% utter shit. I had to dig and dig and google and dig for days to find this.


Videos

Categories
Web Development Web Security

Our programming tools are stuck in the past

Recently I decided to start automating my infrastructure. Before this it had never occurred to me how stuck in the past our ancient tools are.

These days we have the cloud. We can fire instances up in seconds. But to do this we need ways of automating things. Tools such as SSH, SSL, GIT etc. feel stuck in the 1990’s . The 1990’s was a period of time when server admins bragged about how many days/hours their servers had been online. No really that was seriously a thing.

In the 1990’s there basically was 0 automation. The only people automating things were shell scripters and they were seen as genius wizards who casted spells and worked magic.

Automating infrastructure provisioning

I’m not saying automation is impossible with today’s tools, but it is insanely hard. The hardest part is finding accurate information, because reading the docs will do nothing but leave you lost as hell. Most docs read like notes for those who already know how to use it, complete with lack of examples.
I can’t be the only person who is like WTF are you talking about when reading docs.

This is the best you can explain this FFS?

One major problem with automating with today’s tools is the fact they were designed mostly for manual use in a different time period. By this I mean most ask a series of questions that are hard as hell to answer automatically, OR EVEN FIGURE OUT THE SYNTAX TO DO SO.

This is some of the syntax I found online suggesting how to answer the questions. I borked it a little with this command, I later found out.


openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/nginx-selfsigned.key -out /etc/ssl/certs/nginx-selfsigned.crt<< EOF
echo `#US`
echo `#Florida`
echo `#.`
echo `#.`
echo `#$this_ip`
echo `#"akashicseer@gmail.com"`
EOF

The above code is supposed to use Heredoc syntax which creates an infile file and feeds the info to the prompts. It doesn’t work. Not sure if plain echo “value \” would do it or not, this is the syntax I found. I did get something similar working though.

Now I must spend at least another 24 hours googling and trying and digging, because most info you find about linux is wrong.

Apparently it depends on if the script asking the questions expects answers from stdin or somewhere else, file etc. Plus I saw somewhere in the openssl docs something about echo is turned off or something? I’ll post it if I find it again.

You’ve got to be kidding me.

SSL is even more fun. The docs for it are terrible. It gives you no idea of what to use how to use it etc. Purely written for the already initiated. This is a major problem I see everywhere in Technology and programming. You have people smart enough to create something, but they can’t explain how to use what they created in a way that others can just pick up and use. This causes lots of wasted human time.

It shouldn’t take days to figure out how SSH works and how to automate. Days to figure out how SSL works and automate. Days to figure out how xyz works and automate it.

This is now 2021 we need improvements to tools( especially docs) so we can more easily automate things.  Our tools need to give us example files of the questions they ask and better yet a copy of how to answer them. Our tools need to be able to easily be directed to a file to read the answers from. Our tools need to focus on telling users how to use them.

Our tools need help.

Our tools need help

I have another article coming soon on how to automate SSL/TLS certificate and csr creation with shell scripts. The same can be converted to the command line since shell scripts are just Linux commands in a file with some special syntax SOMETIMES.

Categories
Software Development Web Development

Linux bash scripting command substitution aka $(command)

Linux has this syntax that looks like so:
$(command)

This is called command substitution. This allows you to get information about the execution of the command instead of having it it directed to STDOUT aka the terminal screen as usual.

That is very useful actually because you can run a command and store the output in a variable and use it anywhere you want later.

A simple example you can easily play with:

DIR_LISTINGS=$(ls -al)
echo $DIR_LISTINGS

This is so simple you don’t even have to add it to a script you can run it straight in your terminal in any directory you user owns.

More information can be found in this excellent book Linux command line and shell scripting bible page 277

More info about command substitution.

Bash manual reference.