Categories
Web Development

Using doctrine commands in the console with Symfony Framework

Here is the issue. If you follow along the docs for Doctrine when working on a Symfony framework, you will get issues every once and a while.

./vendor/bin/doctrine-migrations status --show-versions

Something missing or not configured correctly etc. That line is copied and pasted directly from the Doctrine documentation . To use Doctrine this way with Symfony, you must do some further configurations to Doctrine.

If you are using Symfony there is an easier way, unless you must absolutely use Doctrine like that. As programmers we love the easier way right…

sweet meme
Symfony makes it easier to work with Doctrine

What I like to do is use Doctrine with the plain php bin/console. But first I had to figure out how to do that.

Step 1, show console commands

To show the console commands available type the following while in the main directory for you app( the one with public, src, var, etc.)

php bin/console list

This will output a long list of all of the commands you can use in the console with php bin/console xxx-name-of-command.
Here is what mine output.

 

As you can see there are a lot of commands and a lot of them for doctrine.

doctrine
  doctrine:cache:clear-collection-region     Clear a second-level cache collection region
  doctrine:cache:clear-entity-region         Clear a second-level cache entity region
  doctrine:cache:clear-metadata              Clears all metadata cache for an entity manager
  doctrine:cache:clear-query                 Clears all query cache for an entity manager
  doctrine:cache:clear-query-region          Clear a second-level cache query region
  doctrine:cache:clear-result                Clears result cache for an entity manager
  doctrine:database:create                   Creates the configured database
  doctrine:database:drop                     Drops the configured database
  doctrine:database:import                   Import SQL file(s) directly to Database.
  doctrine:ensure-production-settings        Verify that Doctrine is properly configured for a production environment
  doctrine:fixtures:load                     Load data fixtures to your database
  doctrine:mapping:convert                   [orm:convert:mapping] Convert mapping information between supported formats
  doctrine:mapping:import                    Imports mapping information from an existing database
  doctrine:mapping:info                      
  doctrine:migrations:current                [current] Outputs the current version
  doctrine:migrations:diff                   [diff] Generate a migration by comparing your current database to your mapping information.
  doctrine:migrations:dump-schema            [dump-schema] Dump the schema for your database to a migration.
  doctrine:migrations:execute                [execute] Execute one or more migration versions up or down manually.
  doctrine:migrations:generate               [generate] Generate a blank migration class.
  doctrine:migrations:latest                 [latest] Outputs the latest version
  doctrine:migrations:list                   [list-migrations] Display a list of all available migrations and their status.
  doctrine:migrations:migrate                [migrate] Execute a migration to a specified version or the latest available version.
  doctrine:migrations:rollup                 [rollup] Rollup migrations by deleting all tracked versions and insert the one version that exists.
  doctrine:migrations:status                 [status] View the status of a set of migrations.
  doctrine:migrations:sync-metadata-storage  [sync-metadata-storage] Ensures that the metadata storage is at the latest version.
  doctrine:migrations:up-to-date             [up-to-date] Tells you if your schema is up-to-date.
  doctrine:migrations:version                [version] Manually add and delete migration versions from the version table.
  doctrine:query:dql                         Executes arbitrary DQL directly from the command line
  doctrine:query:sql                         Executes arbitrary SQL directly from the command line.
  doctrine:schema:create                     Executes (or dumps) the SQL needed to generate the database schema
  doctrine:schema:drop                       Executes (or dumps) the SQL needed to drop the current database schema
  doctrine:schema:update                     Executes (or dumps) the SQL needed to update the database schema to match the current mapping metadata
  doctrine:schema:validate                   Validate the mapping files

I believe the replacement for the failed show-versions command above is this

 php bin/console doctrine:migrations:list

Which will output information in the following format.

doctrine migration list output
Output of the command php bin/console doctrine:migrations:list

If I had migrations they would be listed and those values shown would be in rows.

Categories
Software Development Web Development

How to quickly create a complete Symfony 5+ app with login and password reset

If you follow this quick guide you should be able to have a simple, yet complete Symfony 5.xx ( or 6 soon ) within a few hours or less if you have done this before. There are lots and lots of baby steps.

This is meant to be more of a cheatsheet. If you are totally new to Symfony, try this Free to read symfonycast as an introduction.

If you want to know more about Symfony authentication read this symfonycast, some things have changed in version 5.xx.

I may eventually take the time to write Linux Shell scripts to automate all of this. The permissions are the most painful part of all of it.

I thought about setting up a simple app and saving it to a repository on github to make it even faster and easier. But the truth is the base code is moving so fast, it would be better to just create new apps with the latest Symfony version so you get the latest updates.

So step 1 is decide if you want to use pure composer or the Symfony binary. I like using plain old composer, but with the Symfony Binary you get the added benefit of a local server, security checks.

The local server setup is nice because it saves you time from having to create virtual servers. You will still need to decide how you want to run the other parts of your app like Mysql/Postgre, Redis, Memcache, RabbitMq etc. I’ve been using docker for some of these and installing others. Soon I will be experimenting with running Symfony Locally with MiniKube, which lets  you run Kubernetes locally.  I’ll have to figure that workflow out and share in another article.

Step 1 create the app

Move to the directory you want your app to exist in and type the following replacing with your project name.

 symfony new my_project_name --full

Step 2 fix permissions

Now that the app has been created you need to fix the permissions on the var folder. Change directories to your app directory. This is the one that contains bin, config, public, src, var etc. Then copy and paste this command into the command line ( ctrl + shift + v ) got to add the shift.

HTTPDUSER=$(ps axo user,comm | grep -E '[a]pache|[h]ttpd|[_]www|[w]ww-data|[n]ginx' | grep -v root | head -1 | cut -d\  -f1)

You must also set permissions for future files and folders in this directory.

 sudo setfacl -dR -m u:"$HTTPDUSER":rwX -m u:$(whoami):rwX var

Almost there…

hide the pain meme
Many steps

Now you must set permissions for existing files.

sudo setfacl -R -m u:"$HTTPDUSER":rwX -m u:$(whoami):rwX var

I have to admit I don’t fully understand what is meant by the next permission step of setting the ”
Use the same User for the CLI and the Web Server

I think what it means, and how I got this working is, find the name of the server you are using, mine is www-data.  More info on setting the server name here and create a group for it

To find the user name your server is running as use the following command

ps -aux

Which will list out all of the processes that are running and lots of info about them. Below is an example. As you can see my nginx is running as www-data

nginx name is www-data

What I did is create a group named server-group and I added www-data to that group. I then set that group “server-group” as the group for the file. Here is a link on how to create a group in linux.

Once you have created the group and added your server user name. You now need to change group permissions on the var folder.

Make sure you are in the app directory of your app still, the one with var, src, public etc. then issue this command( change it for your name and group name)

sudo chown -R akashicseer:server-group var

Then you must make sure the new group can read and write to that directory or else you get an error. So you also need to issue this command.

sudo chmod -R 775 var

Now that all of that is out of the way. You can finally add the useful bits.

Install & Configure Doctrine

Before doing anything else you need to install and configure Doctrine as some of the other commands require it.

I like to put the database and other info in env.local instead of .env so the values are not stored in my repository. So first create .env.local For more information about the many configuration files and how they work in Symfony view this.

I prefer listing the database parts instead of a long connection string, it is easier to substitute later. I can use system variables for DATABASE_USER and DATABASE_PASSWORD.  I have also set the app environment variables like this and here too.  And you will want to see what environmental variables your symfony app is using too. I also set the configuration for things like Redis in .env.local

DATABASE_USER=dingleberryFarmer
DATABASE_PASSWORD=aBigPassword

Now you will also need to make some changes to config/packages/doctrine.yaml

dbal:
  dbname: sogi-test
  host: localhost
  port: 3306
  user: '%env(resolve:DATABASE_USER)%'
  password: '%env(resolve:DATABASE_PASSWORD)%'
  driver: pdo_mysql
  server_version: '8.0.21'
  schema_filter: ~^(?!t_)~
    # IMPORTANT: You MUST configure your server version,
    # either here or in the DATABASE_URL env var (see .env file)
    #server_version: '5.7'

Here I add the database name, user name, password (from the .env.local file) the driver etc. The Schema filter is so that doctrine doesn’t mess with my custom tables which have to start with t_ .

To learn more about Doctrine 2 with Symfony 5 I suggest this free to read symfonycast on doctrine.

Creating the User classes

For this you need to use makers. First install the maker bundle with this command.

composer require --dev symfony/maker-bundle

Now list all of the available makers with this command.

php bin/console list make

You will see one named “make:user”. Now type this command and it will ask some questions.

php bin/console make:user

After you answer the questions, it will create a user table, entity and repository class for you.

cool stuff meme
Cool stuff man.

Here I went with the default settings.

symfony make user image
How to create the users

Experiment to find out what the others do.

Create the Authentication system

I learned this the hard way by creating the registration before the authentication. This creates the login/logout forms and supporting classes. Use the following command to create the authentication system.

php bin/console make:auth

This will ask you some questions. I named everything “Loginxxx” So it would match. I first had the defaults but it was too confusing with the name mismatches.

symfony make authentication image
make:auth asks some questions

Create the registration system

Next create the registration system with this command :

php bin/console make:registration-form

This command will ask questions I mostly went with the defaults except for requiring an email to be sent to verify the email.

symfony make registration image
Questions symfony maker asks while creating the registration system

The options for redirect after registration suck because there are no other routes yet. This will be changed later.

kitten stay strong meme
See lots of baby steps

Password reset time

What good is an app if users can’t reset their passwords? Lucky for us Symfony makes this easy too, with another Maker. LOL

Before you can use this maker you must install a bundle with this command.

composer require --dev symfonycasts/reset-password-bundle

Now use this command to create the password reset classes.

 php bin/console make:reset-password

This command will ask you some questions like so

symfony reset password image
Symfony maker will ask these questions

Below where it says next it shows that you should create a migration. In order to do that next we must setup some configs.

symfony reset password image 2
Options after reset password is created

Now follow the advice in the last screenshot and do some doctrine migrations to create the needed tables for all of the Entities you now have.

You can now add more contollers and build your app. After you get another controller and some routes you can change the route to direct users to after they register. I redirect mine to the last page they viewed or to their home page. I’ll have to write an article about that too.

Categories
Software Development Web Development

Symfony 5+ how to create a Maker

The docs do not mention how to create your own maker bundle. It gives a link to some of the Github code of the existing maker bundles for you to view as an example.

you need to figure it out meme
Here is a link, you figure it out. LOL

So when I dug deeper into the subject, I figured out what you are actually doing with a maker is creating a console app. You will know these “Makers” from typing commands on the command line like the following:

php bin/console cache:clear  or 
php bin/console list make

So step 1 is to create a new class in a directory in your “src” directory. I call mine… “Makers”

Then inside your class you need to add some use statements for the classes you intend to use like so.


use Symfony\Bundle\MakerBundle\ConsoleStyle;
use Symfony\Bundle\MakerBundle\DependencyBuilder;
use Symfony\Bundle\MakerBundle\Generator;
use Symfony\Bundle\MakerBundle\InputConfiguration;
use Symfony\Bundle\MakerBundle\Maker\AbstractMaker;
use Symfony\Bundle\MakerBundle\Validator;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;

Here is a link to the Console classes so you can find out what is available.
For the other classes you see e.g. “Symfony\Bundle\MakerBundle\ConsoleStyle  It appears these are located in “maker-bundle/src/class-name/ now. Here is a link to the related classes source code.

Once you have decided what classes you will need, via trial and error based on your project idea, you next need to extend the AbstractMaker interface and add the methods like this.

Below is an example of a maker that extracts table information from a database and creates classes with it. I use these classes for creating custom queries. It is easier than remembering all the fields of a table and what values they can contain etc. The only downside is when changes to a table are made and the column name you used no longer exists, you have to find it in your code. But a good IDE will help you find this. I might add this code to a repo or maybe Symfony one day. This is useful for writing custom queries where you need to know the column names and types etc. You can use this with Doctrine CRUD classes or your own etc.

I made notes on what I think the methods do since there is absolutely 0 documentation that I can find.


<?php


namespace App\Maker;


use App\DbUtils\MysqlConnection;
use App\Schema\SchemaMaker;
use Symfony\Bundle\MakerBundle\ConsoleStyle;
use Symfony\Bundle\MakerBundle\DependencyBuilder;
use Symfony\Bundle\MakerBundle\Generator;
use Symfony\Bundle\MakerBundle\InputConfiguration;
use Symfony\Bundle\MakerBundle\Maker\AbstractMaker;
use Symfony\Bundle\MakerBundle\Validator;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;

class TableMetaMaker extends AbstractMaker
{

    /**
     * Return the command name for your maker (e.g. make:report).
     */
    public static function getCommandName(): string
    {
        return 'make:table-meta';
    }

    /**
     * Configure the command: set description, input arguments, options, etc.
     *
     * By default, all arguments will be asked interactively. If you want
     * to avoid that, use the §inputConfig->setArgumentAsNonInteractive() method.
     * @param Command §command
     * @param InputConfiguration §inputConfig
     */
    public function configureCommand(Command §command, InputConfiguration §inputConfig)
    {
        §command->setDescription('Creates classes with table data')
            ->addArgument('schema', InputArgument::REQUIRED, sprintf('The name of database schema to model '));
    }

    /**
     * Configure any library dependencies that your maker requires.
     * @param DependencyBuilder §dependencies
     */
    public function configureDependencies(DependencyBuilder §dependencies)
    {
        // TODO: Implement configureDependencies() method.
    }

    /**
     * Called after normal code generation: allows you to do anything.
     * @param InputInterface §input
     * @param ConsoleStyle §io
     * @param Generator §generator
     */
    public function generate(InputInterface §input, ConsoleStyle §io, Generator §generator)
    {
        §schema = §input->getArgument('schema');
        §io->success("Now generating Table classes for schema " . §schema);
        §userName = §_SERVER['DATABASE_USER'];
        §pass = §_SERVER['DATABASE_PASSWORD'];

        §mysql = new MysqlConnection(§userName,§pass, §schema);
        §schemaMaker = new SchemaMaker(§mysql,§schema);
        §schemaMaker->setSchemaTableList();
        §schemaMaker->createAllSchemaClasses();

        foreach (§schemaMaker->getTablesList() as §table){
            §io->success("Created Class for table " . implode('-', §table));
        }

    }

    public function interact(InputInterface §input, ConsoleStyle §io, Command §command)
    {
        §io->title('Create the database table classes...');
        §value = '';

        if (null === §input->getArgument('schema')) {
            §argument = §command->getDefinition()->getArgument('schema');
            §question = new Question(§argument->getDescription());
            §value = §io->askQuestion(§question);
            §input->setArgument('schema', §value);

        }
        §value = §input->getArgument('schema');
        §io->success("Table classes were made from schema " . §value);

        /*§input->setArgument('schema', §io->ask(
            'What is the name of the database schema',
            null,
            [Validator::class, 'notBlank']
        )
        );*/
    }
}

As you can see there are some custom classes I am using too. I don’t have the time right now to cleanup and publish all the code. Maybe one day though. I have several other Makers I use. One clones Entities, another creates Repositories based on Entities. Those two I may publish sooner as they are the most useful and would require the least effort to share.

Since all you are doing with a Maker is creating a console app. I’d recommend reading all the links below about that. When you extend

AbstractMaker

Here is an absolute basic class with the basic methods you must have for your maker to work.


<?php

namespace App\Maker;

use Symfony\Bundle\MakerBundle\ConsoleStyle;
use Symfony\Bundle\MakerBundle\DependencyBuilder;
use Symfony\Bundle\MakerBundle\Generator;
use Symfony\Bundle\MakerBundle\InputConfiguration;
use Symfony\Bundle\MakerBundle\Maker\AbstractMaker;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;

/**
 * @method string getCommandDescription()
 */
class TestMaker extends AbstractMaker
{

    public static function getCommandName(): string
    {
        // TODO: Implement getCommandName() method.
    }

    public function configureCommand(Command $command, InputConfiguration $inputConfig)
    {
        // TODO: Implement configureCommand() method.
    }

    public function configureDependencies(DependencyBuilder $dependencies)
    {
        // TODO: Implement configureDependencies() method.
    }

    public function generate(InputInterface $input, ConsoleStyle $io, Generator $generator)
    {
        // TODO: Implement generate() method.
    }

    public function __call($name, $arguments)
    {
        // TODO: Implement @method string getCommandDescription()
    }
}

I don’t work on Symfony core and there is no documentation so I can’t tell you what these methods do. However, you can look at all of the code for the makers and see what is done in them.

I do most of my work in the generate method. For getCommandDescription() method I do this

public static function getCommandDescription(): string
{
    return 'Clones a Doctrine Entity class.';
}

Additional info and links

I had to google and dig to find this  which explains the console command arguments and options.

Here is another link to the docs for the console commands you can use with Makers.

Here is yet another link to more information about the console and how to use it. This is to the console component itself. This link has most of the other helpful console related links to the bottom of the article. There looks to be about 20 links at this time. These are the most helpful links so I won’t repost every one of them here.

Here is yet another link about the console arguments and options.

And here is a link that explains the QuestionHelper class you use to ask questions.

Here is a link explaining the FormatHelper class used to format console output. This one is useful if you want to print messages in blocks with background colors.  If you want to change the color of the actual text you need to read this.

You can even display a clickable link to the user in the command line.

Categories
Uncategorized

Typing lag on linkedin sucks.

Ever try typing anything on Linkedin? Like not on your phone, so you type faster than 10 words a minute or a single entry to the input?

WOWWWWWWWWWWWWWWWWWWWWW is it laggy and slow.

Typing lag on linkedin

So laggy and slow I often have to just stop trying for up to a minute while Linkedin code does WTF ever it does. What are you people doing anyways recording every damn keystroke? Why does it constantly pause while I am typing? I can’t figure out what is going on to make it so slow other than some sort of key logging or trying to save the comment or searching for something?

Don’t believe me? Watch this. I got so damn tired of the lagginess that I screen captured this mess. At about 29 seconds you can see it looks like I am not typing. I am still typing it is just not responding and lagging.

Categories
Uncategorized

Php how to use array_map with an anonymous function

In PHP you can use array_map() and the other array related methods like array_filter() with either a named function or an anonymous function.
I didn’t see any examples in the PHP docs about how to use an anonymous function, so I tried it and here is how to do it.
Below is the array_map function definition from the PHP docs.
array_map(?callable $callback, array $array, array ...$arrays): array
As you can see it takes a callable/callback function. This means you could create a function and supply it to this method. And that is a great way to do this… unless what you need to do in the function is very small.
For example I needed to remove the word “Repository” from every file name. I globbed the file names from the directory then I use this function


private function getRepositoryNames(): array{
$values = array_map(function ($val){
return str_replace('Repository', '', $val);
}, $this->_repositoryNames);

dump($values);
return $values;
}

Look closely and you will see the anonymous function. It is the first argument to the array_map() method. The second argument is the actual array I want to work on here. As you can see this would quickly become hard to read if the function were more than a few lines of code. You see this a lot more often in languages like Javascript and Scala.

As you can see I am simply using str_replace() If I needed to do much more it would be better to have a separate named function instead of an anonymous one for several reasons. The #1 reason to separate out the function is for readability purposes. The #2 I can think of relates to #1, maintaining or changing how the function works. But mostly if it is long separate it out so it is easier to read the code.

How does it work?

Basically the way array_map works is the second value you supply, an array, is looped over and supplied to your function one at a time and your function operates on it and returns it to array_map. When array_map is done looping over all the values it returns the new array. You can read more in the PHP docs here.

Categories
Web Development

PHP how to convert Pascal Case to Underscore case

I recently ran into a situation where I needed to convert Pascal case Entity names in Symfony to underscore names like so :

UserSettings to user_settings

I was trying to convert the Entity names into table names  for a Symfony maker bundle that clones Entities.

First off you will need to understand the basics of Regular Expressions. Here are some helpful links.

Here is a really nice introduction to regular expressions.

A general guide to regular expressions here.

MDN guide to Regular Expressions with Javascript.

A really nice online Regex testing website.

Okay now that that is out of the way. You will need to use PHP preg_match_all() as it returns all the values it finds given the pattern you supply. I was confused, at first I was trying preg_split(). Who knows WTF that is actually for, it returns empty strings given the same regex. If you give it all the arguments it can take, it gives you some nonsensical garbage error, that no amount of searching will help you with. Yay PHP.

The regular expression that does what we need is actually quite simple. It is as follows :

$pattern = '/[A-Z][a-z]+/';
This will find the words in a string that follow the pattern of Capital letter followed by one of more lower case letters which UserSettings contains 2 words by this pattern.

Who needs consistency

PHP is wildly inconsistent so preg_match_all() doesn’t actually return a value. Like you can’t just say
$value = preg_match_all();

No, this is a time when PHP wants to make things Exciting and turn things upside down. Instead of you getting a return value that you store in a variable, you supply a variable name for PHP to store the result into in the method call itself. Like WTF?

preg_match_all($pattern, $newEntityName, $nameParts);

Here the 3rd argument “$nameParts” will hold the results of the function execution, “$newEntityName” is the PascalCased value I want to split up.

Almost there now the code has our Pascal case split into multiple words. These words need to be joined by an underscore _ and lower cased in order for “UserSettings” to become “user_settings”

Here is the final example all lines of code needed.

private function getTableName(string $newEntityName): string
{
$pattern = '/[A-Z][a-z]+/';
preg_match_all($pattern, $newEntityName, $nameParts);
$tableName = implode('_', $nameParts[0]);
return strtolower($tableName);
}

Above you can see that I stored the values returned from preg_match_all() inside a variable named $nameParts.
Then I used implode to connect the “$nameParts” by an underscore which gives me “User_Settings” instead of calling the strtolower twice, it is easier and more efficient to build the entire string then send it through strtolower and return it.

mr rodgers how it is done meme
And that is how it is done!!!