Categories
Software Development Web Development

How to remove unused or broken docker container images.

Sometimes we make mistakes. When first learning docker we probably make many mistakes and end up with tons of unused docker images.

If you are on Linux like me, you won’t have a desktop dashboard like Mac and Windows get, so things are harder. To see a list of what images you have created you use the following command

docker ps -a

That command will output something like this.
It will show the CONTAINER ID, IMAGE, COMMAND etc. as you can see. To delete an image you use the container id with docker rm like this.

docker rm b5f8fae52bce

I’ve seen older internet posts using the IMAGE value but I had no success with that method. I am guessing something changed. I didn’t even see an example of this I just tried it. I don’t see any mention of this in the docs either. But it works. This part in the getting started intro actually explains it. I think something did change.

Docker container rm documentation.

Docker rm documentation.

Categories
Resources Software Development Web Development

HTTP headers and caching resources.

Resources all about HTTP headers and caching.

Caching tutorial -> great article to start with, explains all the basics of caching.

Hypertext Transfer Protocol (HTTP/1.1): Caching  rfc spec

Categories
Resources Web Development

How to create a cookie in Symfony 5.0+ and render a template in a controller

First what I wanted to do was create a cookie in a Controller and display a template at the same time. Sort of like when a user visits a page you set a page count or something. The documentation doesn’t really show an example, you are expected to know it via “common sense” apparently according to one smartass.

There is more than one way I have discovered over time. Apparently you can use render the same way I show using renderView.

Below is the Symfony Cookie class create method comment/documentation. This is all of the values you can supply when creating a cookie.

 /**
     * @param string                        $name     The name of the cookie
     * @param string|null                   $value    The value of the cookie
     * @param int|string|\DateTimeInterface $expire   The time the cookie expires
     * @param string                        $path     The path on the server in which the cookie will be available on
     * @param string|null                   $domain   The domain that the cookie is available to
     * @param bool|null                     $secure   Whether the client should send back the cookie only over HTTPS or null to auto-enable this when the request is already using HTTPS
     * @param bool                          $httpOnly Whether the cookie will be made accessible only through the HTTP protocol
     * @param bool                          $raw      Whether the cookie value should be sent with no url encoding
     * @param string|null                   $sameSite Whether the cookie will be available for cross-site requests
     *
     * @throws \InvalidArgumentException
     */

If you create a cookie like this :

$response->headers->setCookie(Cookie::create('foo', 'bar'));

Then the cookie will only live/exist until the user closes their browser(unless your browser restores from your last session). You must supply an expires time to make it persist beyond closing the browser. Providing an expires time gives you better control over when the cookie expires due to the above mentioned browser restore issue which will restore cookies that should have died on browser close.

You can also create the cookie then pass it to setCookie() like this.

 $response = new Response();
        $expires = time() + 36000;
        $cookie = Cookie::create($cookieName, $cookieValue,  $expires);
        //$cookie = $response->headers->setCookie(Cookie::create('foo', 'bar'));
        $response->headers->setCookie($cookie);

        $content = "<html><body><h1>Learning symfony cookie creation techniques?</h1></body></html>";
        $response->setContent($content);
        $response->headers->set('Content-Type', 'text/html');
        return $response;

Here I set the expires to a number,  time() returns a linux/unix timestamp and I added 36000 seconds or 10 hours to it. This cookie will exist until the user refreshes their page or clicks a link in 10 hours from creation. However long you want it to live you add that many seconds. Or you could create a date using PHP DateTime as you can pass a DateTime object to the expires position. You then use the methods of DateTime to increase the time to a period in the future and pass the DateTime object after calling the methods to do so.

Side Note : in the above code, you can create a cookie without the $response->setContent() call. I do that with the body tag so that the profiler will show up at the bottom of the page for debugging.

That code goes inside a controller method for the requested route by the way. Usually you use the render() method inside a controller to send a response, which renders the template and sends it in a response. You can also use renderView to do the same thing and capture the value in a variable then use setContent or just make the renderView call right in setContent. I know that works. You can also store the returned value from render the same way.  But no matter how you do it, you must return the response object, the very last line. You can find all the methods of the Response class here in the source code.

If you wanted to render a view which requires variables to be sent you do it like this and capture the output of renderView().


 $content = $this->renderView('blog/display_article.html.twig', [
            'title' => $title,
            'article' => $article,
            'tags' => implode(', ', $tags),
            'tagLinks' => $links,
            'edit' => $editLink,
            'affiliateUrl' => $affiliateUrl,
            'backButton' => $backButton
        ]);

Note : do not just use php setcookie or setrawcookie. The reason is they start sending output headers to the browser, which may interfere with how symfony works. You probably won’t notice in a browser, but you may get errors when testing your controllers with functional tests etc.

Personally I created a huge class which extends DateTime which has all kinds of methods for adding days, hours, removing them and doing other math. I’d share it on github but it has bugs since I wrote it way back in version 5 of php in 2012. Some changes were made to DateTime and I haven’t had time to review them all and hunt down the changes that need to be made yet. I’ll probably do it and add it to github eventually.  But for now I use time() + seconds. It’s not the best solution but it works and I only need this one cookie.

And another person found me more hidden docs about cookies, I wish I had this days ago.

As another note. Any values you put in a cookie you must sanitize before trying to use them in any way since users can access and change regular cookie values.

Categories
Resources Web Development

Symfony 5 how to clear the cache

I can never ever remember where I see anything ever I read entirely too much about entirely too many subjects. I mostly use this site as my own personal google.

To clear all caches
php bin/console cache:pool:clear cache.global_clearer

Symfony docs link to more info.

Categories
Resources Web Development

Doctrine migrations links

I have trouble navigating the Symfony documentation so I create lists of links here so I can just easily find what I need again later.

MigrationsBundle link and info -> shows how to run migrations and all of the doctrine migration related commands

Doctrine Symfony overview -> more information about doctrine and migrations.

Categories
Software Development Web Development Web Security

How to create ssh keys for admin user login without passwords

The idea is to have a way for an admins to SSH into a server without having to use passwords. This adds a level of security to your server setup. Without private keys you have to enter your user name and password. This can be less secure than generating SSH keys and adding your public key to SSH, plus with keys you don’t have to remember passwords.

First you need to generate the SSH keys. I prefer the ed25519 algorithm which is a newer one. You can get more info here.  

The code to create an ed25519 ssh key in the current users .ssh directory will look like this.


ssh-keygen -f ~/.ssh/key-name -t ed25519  

The -f flag tells ssh-keygen the name of the files you want to create. The above command would create key-name(private key) and key-name.pub(public) key, in the current users .ssh directory. The ~ is a Linux shortcut meaning /home/current_user/ so you don’t have to type all that.

The -t flag tells ssh-keygen what type of algorithm to use. If you don’t specify the -f flag and give the file a name, then both files are output in the current users .ssh directory as ed25519 and ed25519.pub

Categories
Software Development Web Development

Where does symfony php framework hide the errors?

I kept saying this over and over and looking everywhere for answers. I finally found this page through googling tons of terms and combos until I found it. I kept thinking my errors would be in the logs I set in my Nginx configs, or even in my PHP configs. But they were continually empty, I was going insane. I seriously blew up on twitter.

BLOWING UP

I thought Symfony was simply suppressing or not passing the errors to Nginx. So the logs are located at the place in the link above from documentation and not in the location you set in the Nginx configs or PHP configs.

By default, log entries are written to the var/log/dev.log file when you’re in the dev environment. In the prod environment, logs are written to var/log/prod.log

What made this confusing was, the docs don’t have a logical link flow when you are reading them trying to learn Symfony. You later find the SymfonyCasts which are better. But what makes it  most confusing is in the docs about configuring Nginx, it even shows the following.

    error_log /var/log/nginx/project_error.log;
    access_log /var/log/nginx/project_access.log;

To me this was showing how to set the error log. This does nothing by the way, not unless nginx itself has an error I guess.

For months I’ve wondered WTF, where are my error logs. I kept putting my app in dev mode so I could debug it via the browser.

Don’t do that!!!!!

To be honest getting Symfony working with Nginx is a pain in the ballsocks. The reason is, Nginx doesn’t pass environmental variables through to php scripts like Apache does/can. If you want that kind of fancy feature you must hack nginx up and use some perl script or something similar. Otherwise with Nginx you must set the environment variables twice, once in nginx and once in shell.

Why would you do that?

Why two locations? Yeah this really angered me and blew my mind at first too. As mentioned above Nginx doesn’t have any easy way to pass the environment variables you set at the Linux server level. This is important with Symfony because you often need to run things like Doctrine on the command line.

So I was setting my Nginx Environmental variables, the app would see them just fine. I’d go to run doctrine or tests and BOOM missing environmental variables like WTF? Or I would set them in the Linux environment, view them with printenv load the app in the browser and Nginx didn’t pass the values to my script. It took a lot of googling to figure that out with lots of trial and failure. To make matters worse, you have to change the environmental variable names in order to run tests so that symfony loads them, otherwise it hides the values.

WTF is happening?

I then found out through experimenting that you had to set the variables for the command line in the Linux environment too. How to permanently set Linux environmental variablees covers how to do that. It’s easier to just Bash script or ansible the entire process with Hashicorp packer than to try to manually maintain it all, setting vars in two different places etc.

So for months I’ve been going insane trying to find my error logs. Today I found the error logs.

Problem solved

 

Categories
Software Development Web Development

How to switch users in Linux Bash Shell script and execute multiple commands as different user

If you search you will find different answers to this. You can do this in multiple ways, here I will talk about 2 ways, single command and multiple commands.

First the idea is to switch from say root user to a named user you created or was created for you on your Linux server to run commands as not the root user. The reason you want to do this is so that everything isn’t owned by the root user. Or you are installing something like PHP Composer which barfs on you if you run it as root user.

You will see some saying to use su others saying to use sudo (some bs options etc.) You will also see really wrong answers on Stack. I have no idea why you would use sudo over su, you can google that. But I do know that su switches users. Here is an article goes into more detail of su vs sudo and when you use both.

Single command syntax

So the first way is to run a single command directly inline. If you are the root user you simply use su The syntax to do so is as follows:


su - username "commandToExecute [command options and arguments]"

It has been my experience that the Double ” Quotes are required or else the shell gets confused. You may be able to use single quotes if you don’t use any variables within the quotes.

Multiple commands syntax

To more easily issue multiple commands or long commands you need to use Linux heredoc syntax.
Heredoc uses <


su - $username <<SHT
     cd $serverDir
     php $composerFile install
SHT

Like I said you can use any Delimiter you want. It is tradition to use all caps for the word, it makes it easier to spot. The ending word (EOF here) has to have no spaces or words before it. You can list any number of commands within that syntax and all will be executed by the user.

NOTE: After the ending EOF the shell returns the user to whatever user you were/are logged in as before the lines of code. If you are logged in as root, you are returned to root. Also when you issue the su command you are moved out of the directory you are in. That is why I used cd to move back to the directory I needed to be in.

More links

More info about changing users on stack here.

Here is a link to heredoc syntax explanation and examples

More information and examples about heredoc in bash

Bash how write large amounts of text to a file

Categories
Software Development Web Development

What does prototypical Javascript look like?

Way back before modern times, like 10 years ago. Javascript had a much funkier way of defining objects. It was called prototypical inheritance. This is still how Javascript works, the classes, modules etc. were all recent additions to the language to make it easier to work with. It is not a very fun way to program because it is like looking at a GIANT JSON more than a class with methods.

So what did/does prototype inheritance look like? Well this…



function JsCollection() {
    this.jsObject = new Object();

}
JsCollection.prototype = {
    constructor: JsCollection,
    addNamedProperty: function (property, value) {
        // only add the property if it doesn't exist, return true if it was created
        //return false if it was not, to allow for testing before adding a new property
        var returnBool = false;
        if (!this.jsObject.hasOwnProperty(property)) {
            returnBool = true;
            this.jsObject[property] = value;
        }
        return returnBool;
    },
    getElementCount: function () {
        var elementCount = 0;
        //loop through the object and add to the count
        for (var elem in this.jsObject) {
            //only add to the value if it is part of collection
            //https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Working_with_Objects
            if(this.jsObject.hasOwnProperty(elem)){
                     elementCount++;
            }
           
        }
        return elementCount;
    },
    getElementValue: function (property) {
        var returnProp = null;

        if (this.jsObject.hasOwnProperty(property)) {

            returnProp = this.jsObject[property];
        }
        return returnProp;
    },
    removeNamedProperty: function (property) {

        if (this.jsObject.hasOwnProperty(property)) {
            delete this.jsObject[property];
        }
    },
    changePropertyValue: function (property, value) {

        if (this.jsObject.hasOwnProperty(property)) {
            this.jsObject[property] = value;
        }
    },
    getAllNamedProperties: function () {
        return this.jsObject;
    },
    namedPropertyExists: function (property) {
        var propExists = false;
        if (this.jsObject.hasOwnProperty(property)) {
            propExists = true;
        }
        return propExists;
    }
};

//var objProps = obj1.getAllNamedProperties();
// how to loop through object properties 
//for(var prop in objProps){
//    console.log("Property is " + prop + ' Property value is ' + objProps[prop]);
//}

As you can see this is an object with functions in old fashioned Javascript syntax. This was too confusing of a syntax for most people, plus you had to learn the inner workings of Javascript and how prototypical inheritance works.

I won’t try  to explain it here as it is pretty complicated to wrap your head around. Some videos may help better than an article.



Categories
Software Development Web Development

How to configure php 7.4+ on Linux Ubuntu 20.04+

Configuring php can be confusing. PHP uses multiple configuration files, but the main ones are both named php.ini. You see with php you can have separate configurations for the app and the command line aka cli.

Why would you do that?

This may sound stupid at first but it is due to the fact that your command line is a different environment from your app environment.  In Linux for example each user of the cli has their own environmental variables. These variables get passed to the php cli. This can drive you insane on linux. You think you set the configurations correctly, app works, try command line and BOOM all messed up.

But I set the damn configs WTF?

The two main files are located at

  1. /etc/php/7.4/cli/php.ini
  2. /etc/php/7.4/fpm/php.ini

Yes they have the same exact name. Yes they contain the same thing. However, one is used for the command line (cli) and the other is for apps (fpm). So you must set the settings in both. If you wanted to use 1 file for both you might be able to remove/rename 1 file such as the cli, set the fpm file. Then use a symlink from the fpm file to the cli directory. It would require some testing but it will probably work.

P.S. The php configuration file is the largest you will ever in your life see. I’m guessing a few thousand lines, it feels like that, but there is a ton of commenting and documentation to help you complete with links.