When it comes to CLI applications, the most common way of performing configuration is
through the use of either CLI flags or environment variables. However, in some applications,
using these two approaches is sometimes quite tedious to do, especially if an application
requires more configuration than what's able to easily be achieved using CLI flags or environment
variables alone. Therefore, one common approach to performing configuration or advanced configuration
with CLI applications is to make use of what's known as a config file, which is what many more
advanced CLI applications end up doing. One such application is Hugo, which allows you to specify
a configuration file, either in the form of .toml, .yaml, or .json as the default site configuration. Then when you go ahead and run this command, you can actually pass in the dash dash config flag
in order to specify the configuration file. You can also specify multiple files in as well. Therefore, this is something that's worthwhile knowing how to achieve when it comes to your own
applications. There is a package that makes this quite easy to achieve, which is actually what Hugo
uses called Viper, which actually happens to be by the same creator as the Cobra package, hence the name. Viper works very well with both Cobra and some of the other packages we'll take a look at in the next
module in order to be able to provide configuration files, as well as being able to also perform additional
configuration through the use of environment variables. In any case, as with most things in
this course, we're going to take a look at how we can actually support config files ourselves before
then looking at third-party packages in the next module. So let's go ahead and actually implement it
into the current application that I have here. In order to do so, we're first going to need to define
a configuration file. Here we can actually choose any type of file that we want, be it a .json,
a .pkl, which is a pickle file not seen very often in the world, a .toml, or a .yaml. In my case,
I'm going to go ahead and choose the yaml encoding, as I find it's the easiest type of configuration file
to write. That being said, I do think toml is slightly superior. However, personally, I find yaml a little
bit easier, mainly because I've written so much of it. So to begin, here we have our config.yaml file,
and we need to go ahead and specify some configuration parameters that we want to load in. First things first, let's go ahead and define, say, a database URL, which we can specify as
PostgresQL, and we'll do user password at host 5432, and we'll just do app. Next, let's go ahead and
define some rate limiting properties. So rate limiter, rate limiter, yeah, we'll do rate limiter. Actually, we'll just do this rate limit, and we'll set the limit to be 10, and the duration to be one
minute. This is going to be passed by the time.duration type, and we can specify a number of
different strings. So we could do one second, one hour, one millisecond, etc. We're going to do one
minute in this case. And lastly, let's go ahead and say blocked IPs, and we'll go ahead and pass in a
number of different IPs here. So we could do 192.138.134.1.1, and we'll just go ahead and do
another random IP. So 208.9.123.1. Apologies, if you have this IP, you will be blocked. Okay,
with that, we managed to define our configuration file, as well as passing in a few different
configuration options. Most of these are actually related to a website, but this will work for a
CLI app as well. With our config defined, the next thing we need to do is to load this into our
application. Therefore, to do so, we need to create a config struct type, which will unmarshal the YAML
file in two. However, rather than doing this inside of the main package, or the main.go, instead,
I'm going to go ahead and do this inside of a config package, specifically the config go inside of the
config directory. So let's go ahead and create this, creating config directory and the config.go file,
and we'll open it up. First, defining the package name of config. This is typically how I like to do
configuration files when it comes to Go applications, as I think it provides a nice way of pushing the
configuration into its own package that I can then test. In any case, with the config package defined,
let's go ahead and create a config struct type as follows, and we'll specify the fields we want inside
to represent the fields we've defined in the config.yaml. So first things first, we want to do a
database URL, which is going to be of type string, and we'll give it the struct tag of YAML, and we'll
set this to be database URL. This will then, this means that this field should now be unmarshaled into
the database URL field of our config type. Next, let's go ahead and define a rate limit type. In fact,
we actually need to define a new type for this, so rate limit config we can do, and this will be another
struct type. Here we'll have a limit, which is going to be a integer. We can have it as an unsigned integer. I'm going to leave it as integer for the moment, just because user input can sometimes be a little
funny, and we'll specify this as limits. Then we'll specify a duration, which as I mentioned is a time.duration. This will automatically be passed when we unmarshal, which is one of the nice things of
unmarshaling with Go. This is going to be a duration. Then for the rate limits, we'll specify a rate limit
config. Some cases you may want to have this inside of another package, so if we had a package of
rate limiter, we could have rate limit.config. This tends to be how I like to nest configurations when
it comes to larger scale applications. However, this will work for the current demonstration. Okay, rate limits, rate limit config. We need to pass in the yaml struct tag, which is going to be
rate limit. And lastly, we want to pass in a blocked IPs. So we can do blocked IPs, and this will be a
slice of string, and it's going to be yaml blocked IPs. With our configuration defined,
and the config struct defined, the next thing we need to do is to load this in. To do so,
I'm going to create a new function called load config, which will take a file path in as a string,
and it will either return a config or an error. Then let's just go ahead and set a default return
value for the meantime, and we'll set this as follows. Now all we need to do is open up the file,
check to see if it exists. If it doesn't, we'll return an error, and we can then just go ahead and
unmarshal it using the yaml encoding package. So to begin, let's go ahead and open up our file
using the os.open function, which returns a file and an error, and we'll just pass in the file path
as follows. Then we can do a simple if error check, and if an error does exist, we'll just return an
empty config and the error. In the next lesson, we'll take a look at how to perform error wrapping using
the fmt.error function, which is how we can perform more advanced error handling. In this case, we're just
going to return the error as is, although I do always recommend wrapping errors when it comes to your
own code if you're returning multiple errors inside, which we will actually be doing so. So I'm actually
going to go ahead and change what I just said, and we'll wrap the error here. Failed to open file, and
having the failed here is kind of redundant. So let's just go ahead and say open file, and we'll import the
fmt package. Okay, now with our file created, let's go ahead and actually do a defer on the close, because
we should always be closing files. Now all we need to do is unmarshall the contents of this file into our
configuration struct. So let's go ahead and define a config struct as follows. Then we can use the yaml
package, which we're going to need to import. This is go pkg.in forward slash yaml.v3. So let's go ahead and
go get this because we're inside of a go mod. So go pkg.i slash yaml v3. Additionally, if your editor
automatically imports, I'm going to show you another way how we can actually get this into our package. So in my case, my editor will automatically add this package to my import list for this file. However,
as you can see, my go mod does not provide the package currently. Well, the easiest way to do this
rather than typing out the package name is to just go ahead and do go mod tidy. As you can see,
it's now finding the module for package go package.in. And if I go ahead and cat the go.mod,
it now exists inside of my dependencies list. However, as you can see, my editor is still
complaining. So just do an lsp restart and everything is working. Now it's complaining because I'm not using
it. Great. So yaml.new decoder, because we want to decode from an io.reader, which is going to be the
file and we'll decode into our interface, which we need to pass in as a pointer because it's going to
modify the original value. This decode method returns an error. So we're going to want to handle
that. We could capture the error using the following syntax. However, when it comes to operations that
return just an error, I like to kind of do an if error check all in a single line, which you can do
as follows. If error, returning it to the result of the function or method and then do the error is not
equal to nil. This one liner does both the assignments or capturing of the error value as
well as performing the error check. You may be thinking, why not just do if is not equal to nil. Well, you still want to capture the actual error value in order to be able to propagate it up to
the caller or to be able to log on it. So in our case, we just want to go ahead and propagate this up
to the caller and we'll wrap it using the format.errorf function. So decode for some reason is
the decode is the one that failed. So we're just marking it with some additional context. All we need
to do now is instead of returning the config, empty config, we can just go ahead and return our decoded
config. So everything should now be working. Let's head on over to the main.go file and we can go ahead
and do say config is equal to config load config. And for the moment, we're just going to go ahead
and hard code this into config.yaml. And of course it returns an error, if error, and we'll just do the
log.fatal line as usual. We don't need to return here. I'm going to set log.setflags to v0 as well,
just because I want to actually check this. And we can go ahead and do format.printline.config. Okay, now if we go ahead and run this code, you can see our config is being printed out to the console
and we've managed to load it in. Pretty cool. Additionally, we also now have access to all of
the individual properties. So we can go ahead and look at the database URL using it within our code. Let's say if we were connecting to the database as follows. And we also have the other two properties
we set as well. So the rate limiter, we have the access to the time.duration and the limits,
which we could use for rate limiting or configuring a rate limiter. And of course we have our blocked
IPs, which we can also now iterate over for IP equals range blocked IPs. And we can just go ahead and
actually print this out. So blocked, let's say, and we do IP as follows. And if we go ahead and run this,
you can see we're now printing out the values we've defined inside of our config.yaml. As I mentioned,
these configurations are more related to web applications, which is typically where you will
find configuration files. Although of course, they can still be more geared towards CLI applications. For example, one of the applications we will be building later on in this course is going to be a
CMS system. And being able to pass in the database URL or assign it through an environment variable
is very useful to have. In fact, when it comes to the actual video management system of my Dreams of Code
website, I actually have a manifest file where I define all of the individual videos or lessons by their
slugs and actually point them to their respective video file, which allows me to run my CLI application
to automatically process and upload these files, registering them to the appropriate lesson slug. Okay, with that, we've managed to implement a really simple configuration loading piece of code inside of
the config package. All it is, is defining a base type for your configuration or a representation of your
configuration, and then loading in the configuration by opening up the file and using your chosen encoding
type to unmarshall or decode into the actual configuration struct, or into your configuration type. That being said, there are some additional features that a good configuration loading system should have in place. The first of which is to specify a flag so we can pass in where this configuration file lives. To do so is actually pretty easy. We've seen how to do this in the past. But there is a key challenge I want to add to this, which I'm also going to set you as some homework. To begin, let's go ahead and actually set in the config file path or file name. Config file name, let's go ahead and do, is going to be a string as follows. Then we need to go ahead and pass this into the string dot or flag dot string file. And we'll set this to be config. The default is going to be empty. And we'll just say used to specify config file. Then we'll just do the flag dot parse and our config should now be defined. If we go ahead and specify this as the config file name, everything should work as it did before. If we go ahead and run this, you'll see no such file or directory. And if we go ahead and specify config to slash config dot YAML, you can see we're loading in our configuration file. Pretty cool. However, there is one change I would like you to make, which is to return a default configuration if we do not pass a flag in. So in this case, if I don't pass a configuration file in, I want to detect that we haven't passed anything in and we should return a default configuration. One thing to configure consider is passing in an empty file doesn't necessarily mean we haven't passed a file in. In some cases, if we do pass in dash config, then in this case, I would like it to actually return an error rather than returning a default configuration. So how to do this when it comes to the flag package. Well, the way to do so is to use the flag dot physic function, which allows us to iterate over each flag that has been set by the user. So if we go ahead and specify this and we'll do a format dot print line, printing out the flag dot name. Let's go ahead and give this a prefix so we can see what is happening a little better. Now if I go ahead and run this again, you can see when I pass in the config flag, you can see the flag name of config appears. However, when I don't pass in the config flag, you can see we don't get any values set. Therefore, you can have another variable here saying config flag sets bool and if the flag dot name f dot name is equal to config, if the flag name is equal to config, then config flag name sets is equal to true. Something like that. So using this approach is how you can actually determine and we could do say here format dot print line, let's say was set and we'll do config flag set. Let's just show that this does work. This is flag sets. OK, now if I go ahead and run this, you can see was set false was set true. So you can use this approach to determine whether or not a flag was actually set by using the flag dot visit function. This is kind of tedious to have to do. Fortunately, there are other packages that make this really easy. However, this is how you do it when it comes to the standard library. In any case, once you have the ability to check to see whether or not a flag has been set, you want to do an if config flag set, let's say, then or if not, let's say, then do config cfg equals config dot default, let's say. So adding in a default function to the config package, which will return a default configuration. That's kind of the first homework I would like you to set in order to be able to define what that default configuration looks like and return it. Finishing out the rest of this function. Additionally, I also would like you to add yet another value. So one that will affect both the default configuration. Let's say in our config package, we have a func default. I'm going to go ahead and set this to be just a config return value. And in this case, it will just return an empty one. In any case, the second piece of homework and perhaps the more challenging one is to be able to use environment variables to overwrite the configuration that we either pass in or as default. For example, if I go ahead and pass in, say, database URL equals foobar baz, let's say, or foobar. And if I go ahead and run the config and we'll do config dot YAML. In this case, the configuration URL defined here should be foobar rather than the configuration defined in either the YAML or in the default configuration. This is going to be a little bit more challenging to achieve, but it's good to understand how to do this as it's used often in the real world. But it's also provided by other packages that we'll take a look at in the next module. In any case, once you've managed to achieve that, we'll then move on to the next lesson where we're going to look at some more advanced error handling, similar to what we saw when it came to wrapping errors, but also how we can use error wrapping and error casting in order to have different behavior depending on what the error type is.