Categories
Web Development Web Security

Faking Enumerations with Vanilla javascript

What is an Enumeration?

An Enumeration is a way to create a limited list of options to choose from.This is useful for keeping a list of field names for a form so you can use javascript to animate something for example.

Having a limited list of options is helpful so that you can eliminate bugs due to misspellings (very common in Javascript UI programming).

A limited list also helps so that you can just type and your IDE gives you suggestions to jog your memory of the available options so you don’t have to dive into code.

See the limitations section.

Javascript has no Enums yet

Javascript has no such concept as an enumerated class… yet( the keyword enum is reserved so maybe in the future). Heck it is 2021 and PHP just got Enum classes.  While it does allow class level variables they are defined in the most funky way inside the constructor with this keyword.

Uhm, wait… what?

I say funky because with most other languages you define variables at the class level, then instantiate them ( give them a starting value ) inside the constructor for example. Just be glad you don’t have to use the old syntax What does prototypical Javascript look like?

So to define class level variables in Javascript you need to do so inside the constructor using the this keyword. The reason for this is how the Javascript prototype system works.

class Rectangle {
  constructor(height, width) {
    this.height = height;
    this.width = width;
  }
}
Before Javascript classes… remembering this

Now anywhere inside the class you can get or set the value by this.height or this.width. You can’t set any constants like this though. Constants have to be defined outside the class if you want to use them inside a class, in all methods/functions. You can define a constant inside a function ( aka constructor ) but it is limited to the function in scope just like the let keyword.

But what if you want something like a list of constants or values that can be used? For example I like to keep my form field id’s inside an Enum to be able to easily refer to the field I need, but how can I do this with Javascript?

The answer

The easiest way I have come up with is to NOT USE a class at all. Instead I just use a simple file with a constant set to an object with a list of values. Sure you could just make a list of constants, but there are downsides to that. For one you would need to export them in order to import and use them.

I prefer to create a constant set to a literal object value inside of a single file, like this.

const MEDIA_FORM_FIELDS_ENUM = {
    ALLOW_COMMENTS: 'allow_comments',
    COLLECTION: 'collection',
    CONTENT_RATING: 'content_rating',
    DESCRIPTION: 'description',
    HASHTAGS: 'hashtags',
    PUBLISHED: 'published',
    REUSE_TYPE: 'reuse_type',
    TITLE: 'title'
};

export {MEDIA_FORM_FIELDS_ENUM};

Note the export.

Then I use it like this in my form or form fragment in this case.

Note the import.

import {MEDIA_FORM_FIELDS_ENUM as fields} from "../enums/MediaFormFieldsEnum";

class MediaOptions {
    static getMediaOptions(mediaType) {
        let collection = fields.COLLECTION;
        let comments = fields.ALLOW_COMMENTS;
        let description = fields.DESCRIPTION;
        let hashtags = fields.HASHTAGS;
        let published = fields.PUBLISHED;
        let reuse = fields.REUSE_TYPE;
        let title = fields.TITLE;

        return `
        <div id="media-options" class="container">
            
            <div class="form-group">
             <label for="content-rating" >${mediaType} rating</label>
               <select name="content-rating" id="content-rating" class="form-control" >
                  <option selected value="rating-everyone" id="rating-everyone" >Everyone</option>
                  <option  value="rating-mature" id="rating-mature">Mature</option>
                  <option value="rating-xrated" id="rating-xrated" >Adult rated-x</option>
               </select>
            </div>
         
          <div class="form-group">
            <label for="${title}">${mediaType} Title</label>
            <input type="text" class="form-control" id="${title}" name="${title}">
          </div>
          
          <div class="form-group">
            <label for="${description}" >${mediaType} Description</label>
            <textarea rows="5" id="${description}" name="${description}" 
            placeholder="describe the image in 200 characters" class="form-control" ></textarea>
          </div>
          
          <div class="form-group">
                <label for="${hashtags}" >${mediaType} Hashtags</label>
                <textarea rows="2" id="${hashtags}" name="${hashtags}"
                 placeholder="separate hashtags with space" class="form-control" ></textarea>
          </div>
             
          <div class="form-group">
            <label for="${collection}">${mediaType} Collection</label>
            <input type="text" class="form-control" id="${collection}" name="${collection}">
          </div>
          
          <div class="form-row">Published/visible status</div>
          <div class="form-check">
            <input class="form-check-input" type="radio" name="${published}"
             id="${published}" value="published" checked>
            <label class="form-check-label" for="${published}">
             Published ( visible to others )
            </label>
          </div>
          <div class="form-check">
            <input class="form-check-input" type="radio" name="${published}"
             id="unpublished" value="unpublished">
            <label class="form-check-label" for="unpublished">
             Un-Published ( visible to only you )
            </label>
          </div>
          
          <div class="form-group">
             <label for="${comments}">Allow comments</label>
               <select name="${comments}" id="${comments}" class="form-control" >
                  <option value="followers" >Buyers only/no one/private</option>
                  <option selected value="everyone" >Everyone & Buyers</option>
                  <option value="followers" >Followers & Buyers</option>
               </select>
           </div>
             
          <div class="form-group">
             <label for="${reuse}" >Allow reuse</label>
               <select name="${reuse}" id="${reuse}" class="form-control" >
                  <option selected value="none" id="reuse-none" >None/private (me only)</option>
                  <option  value="free" id="reuse-free">Free</option>
                  <option value="credits" id="reuse-credits" >Credits</option>
               </select>
            </div>
           
        </div>
        `;
    }
}

export {MediaOptions}

That is a lot of code. Note it is HTML inside of a Javascript Literal. I’ll write another article about creating templates with Javascript literals later. For now note how I imported it and used it. I could have just called the fields.OPTIONS but that is longer than a variable name.

I use the above code by importing it into yet another file that builds a whole form but only when called. Like I said I’ll have to write an article about the Javascript Literals, because wow they are handy.

Vanilla javascript might be a little more work, but in the end when something doesn’t work you know exactly why and exactly where to look. And if it is a bug… IT IS YOUR BUG and you can quickly fix it and move right along.

Limitations

Javascript is a real screwy language with lots of limitations and quirks. Many don’t make a lick of sense, unless you have the unlimited free time to dig deeply into the internet to dig deeply into how the Interpreter works.

The above was working fine… until I tried to use it in an Object. When I try to use something like

let stupidObj = { ImageDataEnum.ALLOW_CONTENT_RATING_CHANGE : 'no', 'test' : 'value' };

But this does not work. You will get an error of some type or another. The error I got was something like ” , found expecting ; ” and then a lot of lines of barfarony.

Javascript interpreter was all like…

baby threw up meme
The limitations are real.

So I figured ok maybe store it in a variable and then try that.

const tester = ImageDataEnum.ALLOW_CONTENT_RATING_CHANGE;
let stupidObj = { tester : 'no', 'test' : 'value' };

That doesn’t work either. The interpreter doesn’t interpret the const tester as the tester I wanted to use as the object key so it outputs something different.
The idea here is I want to store all of the key names in a constant and use them later to build an object. This way I for sure know I will spell the key names correctly and I don’t have to remember them all, my IDE can show me the list.  This way I can request some data with AJAX and set the values for an object, then use the object to manipulate the values or use them.

Using the fake enums like this works perfectly fine.


import {JsCollection} from "./JsCollection";
import {ImageDataEnum} from "../enums/ImageDataEnum";
import {Utils} from "./Utils";

class ImageData {

    constructor(imageData) {
        this.imageData = new JsCollection();
        this.setValues(imageData);
    }

    getAllowComments(){
        return this.imageData.getElementValue(ImageDataEnum.ALLOW_COMMENTS);
    }

    getAllowResale(){
        return this.imageData.getElementValue(ImageDataEnum.ALLOW_RESALE);
    }

    getAllowContentRatingChange(){
        return this.imageData.getElementValue(ImageDataEnum.ALLOW_CONTENT_RATING_CHANGE);
    }

    getAltText(){
        return this.imageData.getElementValue(ImageDataEnum.ALT_TEXT);
    }

    getCollectionName(){
        return this.imageData.getElementValue(ImageDataEnum.COLLECTION_NAME);
    }

    getContentRating(){
        return this.imageData.getElementValue(ImageDataEnum.CONTENT_RATING);
    }

    getCreationDatetime(){
        return this.imageData.getElementValue(ImageDataEnum.CREATION_DATETIME);
    }

    getDescription(){
        return this.imageData.getElementValue(ImageDataEnum.DESCRIPTION);
    }

    getFileUrl(){
        return this.imageData.getElementValue(ImageDataEnum.IMAGE_FILE_URL);
    }

    getHashTags(){
        return this.imageData.getElementValue(ImageDataEnum.HASHTAGS);
    }

    getImageFileData(){
        return this.imageData.getElementValue(ImageDataEnum.IMAGE_DATA_FILE);
    }

    getImageId(){
        return this.imageData.getElementValue(ImageDataEnum.IMAGE_ID);
    }

    getLanguageCode(){
        return this.imageData.getElementValue(ImageDataEnum.LANGUAGE_CODE);
    }

    getLastEditDatetime(){
        return this.imageData.getElementValue(ImageDataEnum.LAST_EDIT_TIMESTAMP);
    }

    getPublishedStatus(){
        return this.imageData.getElementValue(ImageDataEnum.PUBLISHED_STATUS);
    }

    getTitle(){
        return this.imageData.getElementValue(ImageDataEnum.TITLE);
    }

    getVisibility(){
        return this.imageData.getElementValue(ImageDataEnum.VISIBILITY);
    }

    setAllowComments(allowComments){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.ALLOW_COMMENTS, allowComments);
    }

    setAllowResale(allowResale){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.ALLOW_RESALE, allowResale);
    }

    setAltText(altText){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.ALT_TEXT, altText);
    }

    setCollectionName(collectionName){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.COLLECTION_NAME, collectionName);
    }

    setContentRating(contentRating){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.CONTENT_RATING, contentRating);
    }

    setCreationDatetime(datetime){

    }
    setDescription(description){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.DESCRIPTION, description);
    }

    setHashtags(hashtags){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.HASHTAGS, hashtags);
    }

    setLastEditDatetime(datetime){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.LAST_EDIT_TIMESTAMP, datetime);
    }
    setImageId(id){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.IMAGE_ID, id);
    }

    setImageFileData(fileData){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.IMAGE_DATA_FILE, fileData);
    }

    setImageFileUrl(fileUrl){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.IMAGE_FILE_URL, fileUrl);
    }

    setLanguageCode(languageCode){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.LANGUAGE_CODE, languageCode);
    }

    setPublishedStatus(publishedStatus){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.PUBLISHED_STATUS, publishedStatus);
    }

    setTitle(title){
        this.imageData.addOverrideNamedProperty(ImageDataEnum.TITLE, title);
    }

    /**
     * used internally to set the imageData values, but JS sucks and has no
     * idea what private or protected is
     * @param imageData
     */
    setValues(imageData){

        this.setAllowComments(Utils.getArrayValue(imageData, ImageDataEnum.ALLOW_COMMENTS, ''));
        this.setAllowResale(Utils.getArrayValue(imageData, ImageDataEnum.ALLOW_RESALE, ''));
        this.setAltText(Utils.getArrayValue(imageData, ImageDataEnum.ALT_TEXT, ''));
        this.setCollectionName(Utils.getArrayValue(imageData, ImageDataEnum.COLLECTION_NAME));
        this.setContentRating(Utils.getArrayValue(imageData, ImageDataEnum.CONTENT_RATING, ''));
        this.setCreationDatetime(Utils.getArrayValue(imageData, ImageDataEnum.CREATION_DATETIME, ''));
        this.setDescription(Utils.getArrayValue(imageData, ImageDataEnum.DESCRIPTION, ''));
        this.setHashtags(Utils.getArrayValue(imageData, ImageDataEnum.HASHTAGS, ''));
        this.setImageId(Utils.getArrayValue(imageData, ImageDataEnum.IMAGE_ID, ''));
        this.setImageFileData(Utils.getArrayValue(imageData, ImageDataEnum.IMAGE_DATA_FILE, ''));
        this.setImageFileUrl(Utils.getArrayValue(imageData,ImageDataEnum.IMAGE_FILE_URL, ''));
        this.setLanguageCode(Utils.getArrayValue(imageData, ImageDataEnum.LANGUAGE_CODE, ''));
        this.setLastEditDatetime(Utils.getArrayValue(imageData, ImageDataEnum.LAST_EDIT_TIMESTAMP, ''));
        this.setPublishedStatus(Utils.getArrayValue(imageData, ImageDataEnum.PUBLISHED_STATUS, ''));
        this.setTitle(Utils.getArrayValue(imageData, ImageDataEnum.TITLE, ''));
    }

}

export {ImageData};

Using the above I can request an images data from the server using AJAX. Then I pass the JSON Object, which was a PHP multidimensional array with keys and values converted to JSON, to this setValues() method.

I can then call the getters to get a value, it will either be a value or an empty string if nothing existed.

With this setup I can set default values when none exist. I also have enums which hold the values I will add as defaults to this later.

So as you can see this is highly useful. This way I can use the Enum anywhere I need a string and I don’t have to worry about spelling. If I do misspell something, I change it one time in one location and I am done. Otherwise I’d have to hunt down all the locations in the text etc. where I had hard coded a value.

But the limitation is Javascript is like WTF is this when you try to use them in Object literals. This is the only place I have found that they don’t work so far.

Dictating like…

It seems like the Javascript interpreter is trying to call a function when it sees the Enum constant reference inside an object literal.  But the rest of the time it works 100% fine. I only wanted this for testing so I’ll move on and not try using it inside object literals for testing LOL.

The work around

So I kept digging until I found a work around, a way to keep using my Fake enums and be able to build an object using the enums as the key names.

You need to use Object.defineProperty() to set a propertyusing the Enum and you get it back using Object.getOwnProperty(). I’ll update this article later when I have the time .

Links

Mozilla Developer Network Javascript class info

Mozilla Developer Network Javascript const info

Mozilla Developer Network javascript literal info

While working on this article I found this excellent article about using Enums in Javascript.

Categories
Web Development Web Security

How to secure individual Symfony AJAX api routes without using API Platform

I am in the process of updating this article entirely. Please stay tuned.

Creating the Symfony route is easy. Checking if the request was sent by AJAX is again easy. But what stops a mischievous hacker from hitting that endpoint and trying to get a list of used emails or something else with a script?

What if you have routes that you want to access with AJAX without API Platform? Maybe your project leader won’t let you use it. Maybe you don’t want to use it. What ever reason you still need to secure your endpoints to some extent. Api platform adds lots of great useful features though so you may want to check it out after reading this article.

For me, some parts of my app are too complex to be dealt with as objects rehydrated etc. My app checks permissions in many places to  see if the user has rights etc. and does creative things with the data, another reason I don’t use Doctrine all the time either. Doctrine and API platform can work together to make most simple actions… even more simple, which can speed up development of simple apps.

With Symfony, standard forms created with the Form Component, your forms are CSRF protected. But, when you are sending an AJAX request to an endpoint without a form how do you protect it?

There is probably some Symfony approved way I am not aware of. There is also more than one way to do this. One is cheap the other is expensive. The cheap I’ll cover first, the expensive I’ll cover eventually.

The cheap I call cheap because it is fast and easy and I’ll describe it below. The expensive I call expensive because it involves more steps + calculations = more cpu = more time etc it may not be much but it is still more. The second way is using JWT’s or JSON Web tokens, which is what API platform does anyways. They are quite complex, take more time to setup and are a little slower than the quick and easy way. JWT’s also increase your bandwidth usage and cause other issues.

The quick and easy I use for API endpoints that do simple things that need to  be as quick as possible, like an endpoint that returns a list of hashtags for a hint list in a form for example. JWT’s take more time to implement and are slower as they require more steps than just checking if two secret strings match.

If you send the whole form you can use a different procedure and use the CSRF string stored in the form. I’ll try to cover that too, somewhere below.

The cheap way

For simple situations where you need to randomly access a route you can do something similar to the CSRF form protection by generating a unique string and saving in a Session cookie and to the page/form.

Where you save the string in the page is up to you, but it should be a hidden element. This element needs a unique ID in the page so that you can access it with Javascript. A hidden input element in a form works great, otherwise use a hidden span element.(use css to hide the element).

When you need to make a request to the route you use javascript to get the value you hid in the element. Make sure it is just the unique string that you fetch not the entire element html or this wont work. Include this string with the data you are sending to the route. You can also store the value in a Secure cookie with HttpOnly set. The way you choose is up to you.

Inside your route fetch the unique string that you sent in your AJAX. Then try to fetch the same unique string from your session cookies. If the string exists and matches process the request.

There are tricks you can try to use with the header like checking the users browser agent. But that is useless as it can be easily spoofed by a good hacker using something like Curl.

This unique string trick isn’t 100% hacker proof. But it makes it a hell of a lot harder.  More on CSRF attacks here.

NOTE

If you are using the Symfony forms with CSRF activated then you can use Javascript to fetch the value of the nonce hidden in the _token input element. However, if your code will make multiple ajax requests, then you might want to create the custom hidden field and generate a new unique string each time and replace it in the custom field.

Step #1 create the field

To create the field add it in the FormType definition like this. The entire class is too long so I’ll show just the add section.


->add('ajaxString', HiddenType::class, [
                'mapped' => false,
                'attr' => ['class' => 'hidden-field', 'value' => $secretString]
            ])

Notice mapped is false so that I don’t get errors.

Step #2 Build the form

Now you build the form inside the Template for the form. Mine looks like this.


{{ form_start(registrationForm) }}
        {{ form_errors(registrationForm) }}
        {{ form_row(registrationForm.email) }}
        {{ form_row(registrationForm.emailMatch) }}
        {{ form_row(registrationForm.plainPassword) }}
        {{ form_row(registrationForm.passwordMatch) }}
        {{ form_row(registrationForm.userAlias) }}
        {{ form_row(registrationForm.ajaxString, { 'id': 'ajaxString'}) }}
        {{ form_row(registrationForm.agreeTerms) }}

        <div class="d-flex justify-content-center">
            <button type="submit" class="btn btn-lg btn-success">Register</button>
        </div>

        {{ form_end(registrationForm) }}

Notice how I have the id : ajaxString line. This is currently the only way to change the ID of a form field in Symfony see How to change the id for a form input in Symfony 5+

Step #3 add initial value

Inside the controller you must add the initial value for the field and store it in a session cookie.

For this I am using a simple class which generates semi random/unique strings. This doesn’t need to be super top notch secure, it is just to make sure the request is coming from a form my app built.

To access the Session Cookie in Symfony 5.3+ you must now use RequestStack instead of Session or SessionInterface for some odd reason. It just makes it more obscure and harder to figure out how to get to sessions.


$session = $this->requestStack->getCurrentRequest()->getSession();
        $secretString = RandomStringGenerator::lowercaseUppercaseNumberString(32);
        $session->set('secretString', $secretString);

 

To check the value in the Controller route endpoint I do like this.


$secretString = $request->query->get('secretString');
        $secretString = DataSanitizer::sanitizeString($secretString);
        $string = $this->requestStack->getCurrentRequest()->get('secretString');

        if ($request->isXmlHttpRequest() && $secretString === $string) {

Note that secretString is the value sent by the AJAX request. This was the value I hid in the form field to use for this purpose.
The other line

$string=$this->requestStack->getCurrentRequest()->get(‘secretString’);

gets the value I stored in the Session Cookie. Then the if statement makes sure the two values match before processing the request. If the two strings match we know that my app built the form, added the string and my Javascript copied the string and sent it to my server. This prevents people from randomly hitting your route endpoints.

&& $secretString === $string

Using the CSRF

I have discovered another way to go about this. Instead of creating your own random string, you can use the CSRF that Symfony already created for this form.

The expensive way

I’ll have to fill in all of the details later for this, it almost needs to be it’s own article. It is expensive because of the time it takes to figure out and implement. Using JWT (json web tokens) with Symfony. Here is a good article I found helpful about the subject. I don’t have time to write one right now.

Links

Here is a good link to Symfony Casts about API Platform. There are many symfony casts here to learn more. I was going to post each but this link contains all of them with pretty pictures and descriptions. LOL

More about CSRF in symfony forms here in the documentation.

Categories
Web Development Web Security

How to add CSRF protection to Symfony 5+ forms

How CSRF works

What you are basically doing with CSRF is trying to make sure a hacker has not created a fake form to gather private information from your user.

You want your app to create a unique string and save it on the server in a session cookie or database, redis etc. Then you add that string to a form in a hidden field. When the form is submitted by the user you fetch the CSRF string you created server side and compare it to the value in the form.

If the values match there was no hacking attempt and you can trust it. If the values do not match then a hacker has probably done something and you should not trust the form submission.

My interweb searches on this subject were not very helpful the other day. So, today I dig into the docs.

on the case detective meme
Time to be a Symfony detective

The first place to look is in the documentation for the security component. That article doesn’t say much about CSRF, but it has lots of links to how the Security system works in Symfony.

In the article above is a link of gold, How to implement CSRF protection.  I didn’t even find this in my  searches for “Symfony CSRF” protection yesterday. I am not sure why, maybe it was further down the list. This article contains some very helpful information though.

You may want to check your current configuration to see if you have enabled CSRF.

So it appears here that just adding to the configuration adds CSRF protection to all forms? Which I find confusing. Like this is truly the only change we need to make?

no way meme
Is it really that easy?

But that is only with forms created using Symfony form tools. If you create a custom form, not using Symfony form system, say REACT, you will need to implement your own CSRF more here.

Here is a good video about Symfony form security and shows how to quickly build an entire app with Symfony

Links

If you need to use PUT with CSRF protection see this SymfonyCast.

Here is a nice SymfonyCast about CSRF with a REST API.

SymfonyCast about CSRF with ReactJs.

Samesite cookie configuration in Symfony docs

Here is a nice Symfony bundle for 2 factor identification with CSRF

This awesome SymfonyCast covers security in  Symfony version 4 but most of it still applies to Symfony 5. It talks more about how using the Symfony form bundle automatically adds CSRF protection when enabled.

Categories
Software Development Web Development Web Security

Authentication vs Authorization what is the difference?

Authentication/Authorization these terms are often confused. Here I will clarify them.

Authentication — Login, proving who a user is one way or another. After a user is logged into a system a session cookie is usually created to re-authenticate the user so they don’t have to login every single page view.

Authorization — Can a user view or access something once Authenticated? Authorization includes things like administration panel access, viewing a users profile or post or media etc.

Categories
Software Development Web Development Web Security

How to create ssh keys for admin user login without passwords

The idea is to have a way for an admins to SSH into a server without having to use passwords. This adds a level of security to your server setup. Without private keys you have to enter your user name and password. This can be less secure than generating SSH keys and adding your public key to SSH, plus with keys you don’t have to remember passwords.

First you need to generate the SSH keys. I prefer the ed25519 algorithm which is a newer one. You can get more info here.  

The code to create an ed25519 ssh key in the current users .ssh directory will look like this.


ssh-keygen -f ~/.ssh/key-name -t ed25519  

The -f flag tells ssh-keygen the name of the files you want to create. The above command would create key-name(private key) and key-name.pub(public) key, in the current users .ssh directory. The ~ is a Linux shortcut meaning /home/current_user/ so you don’t have to type all that.

The -t flag tells ssh-keygen what type of algorithm to use. If you don’t specify the -f flag and give the file a name, then both files are output in the current users .ssh directory as ed25519 and ed25519.pub

Categories
Resources Software Development Web Security

Stupid linux issues.

This is my collection of stupid.

Top of the list Debian/Ubuntu removes apt-key support and doesn’t tell anyone they did it, doesn’t give anyone a single hint as to what to do. No just remove/deprecate shit and don’t tell a single soul on earth. This kind of stupid makes me want to leave the industry entirely. I get so tired of messed up  and undocumented shit that wastes hours and hours and hours of my time. Someone needs kicked for this.

More info and links about the above issue or removing apt-key support. Yarn suggests using apt-key so this means hundreds of millions of people are having this issue or will or could.
Even more info about the stupid ideas from above.

Categories
Software Development Web Development Web Security

How to create ssh deploy keys for github

One issue with creating SSH keys is there are so many ways to do it and no one tells you why they do what they do. A quick search will reveal almost everyone has their own way of doing it.  If you are new to ssh keys I suggest you read this article really quick.

In the github docs they tell you how to create the ssh keys like this

ssh-keygen -t ed25519 -C "your_email@example.com" 

I prefer using the following command instead :

ssh-keygen -t ed25519 -f /home/akashicseer/tests/ssh/file_name -C "akashicseer@gmail.com"

Quick command facts:

  1. ed25519 is basically the newest type of key, it is supposed to be the most secure
  2. -C is for adding a comment to the key. This helps you identify it later in places like ~/.ssh/known_hosts, ~/.ssh/authorized_keys and when you use the command ssh-add -L which prints out your public key info
  3. -t specifies the type of key. The above command tells ssh-keygen to create an ed25519 type of key more info
  4. -f /home/akashicseer/tests/ssh/file_name This tells ssh-keygen where to put the file. If you don’t specify the name then it will use a default of something like id_ed25519 for private key and id_ed25519.pub for the public key. The code above will put the files named file_name (private key) and file_name.pub ( public key) in the folder /home/akashicseer/tests/ssh/ If you don’t specify the full path to the exact folder your keys will be put into your users home directory in the default .ssh location. On Ubuntu this is /home/username/.ssh/

NOTE: for ssh deploy keys, don’t specify a passphrase when you create them or you will have to manually enter it later when Packer or whatever you use runs your provisioning code. That means you won’t be able to automate if you enter a passphrase, because it will ask the terminal user to enter the phrase to do a git clone.

There are different types of ssh keys. If you don’t add the ed25519 part then a regular ssh key of type rsa is created, this is the default type of ssh key. Basically Github documentation is showing how to create a secure type of key to use with your code deployments. You will use this key to clone your repository to your server instances.

Creating the key is only half the battle. You must decide how you will create the key, especially if you are automating deployments. When automating deployments the process becomes very complicated.

First either you create the keys you need in the instance you are creating then use the github api to add them to the proper repo. Or you create the keys on a local development computer and use something like Hashicorp Packer to upload the files to the server instance during creation. The latter is the easiest way especially in automation of the infrastructure.

If you are creating your keys locally and using Packer to upload them, you will need to login to Github and go to the deploy keys section of the specific repo to add your public key. The public key is the one that ends in .pub usually. The easiest way to copy the key value is to use xclip which I mention in this article.

If fully automating the process and creating the key on the actual instance, you must remember to eventually remove older keys. Github lets you have like 50 keys per repo max. If your repo needs to be deployed to many instances, such as a microservice structure you can contact them to get added key abilities. You could also reuse the same key, but that would require keeping the private key in a repo as well which probably isn’t a very good idea at all, since ssh keys are the same as passwords basically.

Also remember this. If you are using deploy keys only to deploy by cloning the repo, then deleting the key after the clone is perfectly fine. You only need to use this key one time to clone (aka deploy ) your code, after this it is useless. You can and probably should create new SSH keys for deployment each time and remove them from Github after you deploy, then delete them from the server instance.

Unless you plan on keeping the same instance up and trying to pull from the repo etc. That is messy. Personally I’d prefer to use Packer to create new instances when I need to and redeploy. This has the added benefit that I can upgrade the instance with security etc., test it, add my app code, test it, then swap over after migrating the database and other files. This is like creating a clean slate every time.

You will also need to know how to add the keys to ssh-agent and use them, which I cover in this article.

Here is a link to a list of resources about ssh.

Categories
Resources Software Development Web Development Web Security

SSL links, videos and other resources.

SSL is a very important subject. All websites/apps should be using it. However the docs will leave you scratching your head saying WTF? So I am creating this long list of resources for anyone else who ever has to learn how to use it.

Articles

First here is a link to the docs – this will cause confusion as nothing tells you how to use the pieces together.  So it is like looking into a box of legos and knowing it builds something but you don’t even have a picture as a hint. The best you can do is use the pieces to build something that doesn’t even resemble the original creation.

OpenSSL quick reference by digicert – a very brief introduction to SSL and how it works

SSL Certificate Security Glossary – list of terms and definitions

How to create a CSR with openssl – shows some of the syntax for the -config file option.

Docs explaining the config file found in the article above bout how to create a csr with openssl

SSL Basics: What is a Certificate Signing Request (CSR)? – Exactly WTF is a CSR

Openssl config file example – openssl docs are pure 100% utter shit. I had to dig and dig and google and dig for days to find this.


Videos

Categories
Web Development Web Security

Our programming tools are stuck in the past

Recently I decided to start automating my infrastructure. Before this it had never occurred to me how stuck in the past our ancient tools are.

These days we have the cloud. We can fire instances up in seconds. But to do this we need ways of automating things. Tools such as SSH, SSL, GIT etc. feel stuck in the 1990’s . The 1990’s was a period of time when server admins bragged about how many days/hours their servers had been online. No really that was seriously a thing.

In the 1990’s there basically was 0 automation. The only people automating things were shell scripters and they were seen as genius wizards who casted spells and worked magic.

Automating infrastructure provisioning

I’m not saying automation is impossible with today’s tools, but it is insanely hard. The hardest part is finding accurate information, because reading the docs will do nothing but leave you lost as hell. Most docs read like notes for those who already know how to use it, complete with lack of examples.
I can’t be the only person who is like WTF are you talking about when reading docs.

This is the best you can explain this FFS?

One major problem with automating with today’s tools is the fact they were designed mostly for manual use in a different time period. By this I mean most ask a series of questions that are hard as hell to answer automatically, OR EVEN FIGURE OUT THE SYNTAX TO DO SO.

This is some of the syntax I found online suggesting how to answer the questions. I borked it a little with this command, I later found out.


openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/nginx-selfsigned.key -out /etc/ssl/certs/nginx-selfsigned.crt<< EOF
echo `#US`
echo `#Florida`
echo `#.`
echo `#.`
echo `#$this_ip`
echo `#"akashicseer@gmail.com"`
EOF

The above code is supposed to use Heredoc syntax which creates an infile file and feeds the info to the prompts. It doesn’t work. Not sure if plain echo “value \” would do it or not, this is the syntax I found. I did get something similar working though.

Now I must spend at least another 24 hours googling and trying and digging, because most info you find about linux is wrong.

Apparently it depends on if the script asking the questions expects answers from stdin or somewhere else, file etc. Plus I saw somewhere in the openssl docs something about echo is turned off or something? I’ll post it if I find it again.

You’ve got to be kidding me.

SSL is even more fun. The docs for it are terrible. It gives you no idea of what to use how to use it etc. Purely written for the already initiated. This is a major problem I see everywhere in Technology and programming. You have people smart enough to create something, but they can’t explain how to use what they created in a way that others can just pick up and use. This causes lots of wasted human time.

It shouldn’t take days to figure out how SSH works and how to automate. Days to figure out how SSL works and automate. Days to figure out how xyz works and automate it.

This is now 2021 we need improvements to tools( especially docs) so we can more easily automate things.  Our tools need to give us example files of the questions they ask and better yet a copy of how to answer them. Our tools need to be able to easily be directed to a file to read the answers from. Our tools need to focus on telling users how to use them.

Our tools need help.

Our tools need help

I have another article coming soon on how to automate SSL/TLS certificate and csr creation with shell scripts. The same can be converted to the command line since shell scripts are just Linux commands in a file with some special syntax SOMETIMES.

Categories
Software Development Web Security

How to use Multiple ssh deploy keys with Github and Git

I came across this issue when automating infrastructure provisioning. I needed a way to pull the repository code for my app in the provisioning scripts. I didn’t want to use the ssh keys I have setup for the entire Github account due to security. I discovered that github has the ability for you to add per repository SSH keys, called Deploy keys.  The docs totally left me in the dark. I had no idea how to do any of this so I had to spend days researching. I decided to write this article to save everyone else hours of time scouring the internet trying to figure out how the hell to do this.

Why use Deploy keys?

Why would you want to use Deploy keys? When automating infrastructure provisioning you don’t want to expose your personal SSH keys. These deploy keys  are going to be used only for cloning a repo, you may be able to use them for other things I didn’t research that not my problem. LOL.

SSH keys when setup correctly, allow a higher level of security than user name and password. Many people are automating by scripting a user name and password, that is BAD. Also if you don’t set a passphrase for the SSH key it won’t prompt for it in the shell terminal. Normally you want your ssh key secured with a pass phrase, but for infrastructure automation we need no pass.

I won’t cover how to automate the infrastructure that would be a series of articles. What I want to cover is how in specific to use multiple SSH keys.

The syntax is wacky as it gets. First off when you are using GIT to pull/push/clone etc. from Github, git is using SSH underneath. So in order to use multiple SSH keys you actually configure SSH not GIT, but git reads the command you type and interacts with SSH on your behalf. Totally confusing. My first few hours were filled with a lot of WTF?

Wait. WTF?

First off the SSH config is stored in your users .ssh directory. On most Linux distributions that is in the user you are logged in as home directory. Basically /home/username/.ssh/ this directory will hold your SSH certs, known_hosts file, config file and others. The ssh config file is always named config and goes in the .ssh directory. If you are logged in as root it will be /root/.ssh/config. Many times when provisioning a server automatically the only user you have at first is root.

Example ssh config file should look like this


   Host hostAlias 
   User git
   Hostname github.com
   IdentityFile=/root/.ssh/id_rsa

Yes it goes in a file exactly like that, no equals, no semicolons no quotes, just 1980’s YAML LOLOL.  The most confusing setting above, which gets more confusing if you read the docs, is Host. Just think of it as Alias. I have no idea why it is even called Host instead of Alias. That threw me and so many others off. I kept putting the same value I had for Hostname. Hostname is the exact name of the host where your repo is, github.com for this example. Identity file is the private key file location.

Another thing to look at is git for the User. You might be able to use other names, but next I’ll show how the name part ties in.

To use the above setting to clone a repo for example you would type the following at the command line.


git clone git@hostAlias:repo-owner-name/repo-name.git .

See the User git and the Host hostAlias. This looks so similar to the regular clone command. For example here is another one of my Github repositories a public one so you can play with this.

git@github.com:AkashicSeer/phphtml.git

This is the default to clone a repo. This has a default name for User of git and a default value for Host of github.com. I haven’t experimented enough yet but I am guessing you can change the name in the configs to anything you want such as Billy and use a command like :
billy@github.com:AkashicSeer/phphtml.git

So back to the question how do you use multiple SSH/Deploy keys with Git and Github?

Like this


   Host hostAlias
   User git
   Hostname github.com
   IdentityFile=/root/.ssh/id_rsa

   Host otherAlias 
   User git
   Hostname github.com
   IdentityFile=/root/.ssh/id_rsa_2

   Host billy 
   User git
   Hostname github.com
   IdentityFile=/root/.ssh/id_rsa_3

Each IdentityFile must be a unique ssh or deploy key, they are the same thing, both are ssh keys.

Then to clone from each for example you would use the following for example.


git clone git@hostAlias:repo-owner-name/repo-name.git .
git clone git@otherAlias:repo-owner-name/repo-name.git .
git clone git@billy:repo-owner-name/repo-name.git .

The format is User@Host:repo-owner-name/actual-repo.git

The dot . I am putting at the end of the clone IS AWESOME.
It tells Git to clone into the current directory and don’t use the name of the repo as the directory name. Basically just clone the damn repo into this damn folder. Without the dot it includes the repo name too. I often just want /opt/app-directory < code in that folder.

AND DON’T FORGET THE SECRET SAUCE

Don’t forget the secret sauce

Now that you have multiple SSH keys you must do some special magic to let SSH know about the keys. For each key you have to tell the ssh-agent it’s name. Basically when SSH does it’s thing your SSH client has to give a list of keys to the SSH agent on the server you are contacting. GIT uses SSH so you must tell SSH where the keys are for your github accounts.

To do this on linux you start the ssh-agent then you add the keys. It is a bit of a pain. First you must start the agent, then you add the key.


#start the agent on linux like so
eval `ssh-agent`
ssh-add /path/to/your/private/key

The value you give to ssh-add command should be the ones you used for your IdentityFile definitions. You will need to add each private key to the agent before it will work.

To test that your setup is working you can do the following and read the output. If there was a problem it will tell you, like it couldn’t find the key.


ssh git@hostAlias
ssh git@otherAlias
ssh git@billy

Running those commands will let you know if everything is configured properly.

BUT IT GETS FUNNER GUYS

The fun is just beginning

All of the 999 things above are still not enough if you want to automate this process.  If you do all of the above and try automating the process, github will prompt you for a passphrase for an ssh key. It won’t be the deploy key either, NO why do that, that would be logical and make sense. What it wants is the passhprhase to the entire account, not the deploy key.

How to fix this?

And there is still, still more, you must chmod the .ssh directory to 600 such as

 chmod -R 600 /home/user/.ssh 

or where ever your ssh files are stored.

You may also need to do the following.

Create a dummy instance. On this instance issue the git clone command. When it asks for the passphrase enter the passphrase for the account that OWNS THE REPO, not the deploy key passphrase which should be empty.

This will add github to known_hosts file. Now use cat to output that info and copy it. You can’t use xclip like I mention in another article, no that’s not allowed for some no brain reason. Once you copy the code from known_hosts create another file on your system called known_hosts. You will need to upload this file along with the ssh deploy keys so that you are not prompted during automated clones.

If there is some sort of openssh setting or a way to do this automatically,  I haven’t found it yet.

If you would like more information on how to create the ssh deploy keys themselves, read this article I wrote.

If you want more information about ssh checkout my list of resources here 

A really good book I found really handy is
SSH Mastery: OpenSSH, PuTTY, Tunnels and Keys (IT Mastery Book 12)