Golang: Paths

Woking with Path and Filesystems using path, filepath, io, os packages in golang.

2022-10-02

Meet Gor

Techstructive Blog @MeetGor21

Introduction

In the 21st post of the series, we will be exploring the file paths in golang, we will be exploring how we can deal with paths. By using packages like os , path , io , we can work with file systems and operating system-specific details. In this section, we will see how to resolve paths, details from paths, extract relative or absolute paths, iterate over file systems, etc.

Starting from this post, it will follow a specific topic in the upcoming few posts which will be covering files and paths. We will be talking about dealing with paths and files in golang. This post is just about working with paths.

Resolving and Parsing Path

In golang, the os and the path packages are quite helpful in working with paths. We use the path\filpath package specifically for working with paths and file structures.

Get the current working directory

To get the path for the current working directory, we can use the os.Getwd() function. The function returns a-ok, an error-like object if the working directory exists it will return the absolute path to the directory else if the path is deleted or corrupted while processing, it will give an error object.

package main

import(
    "os"
    "log"
)

func main() {

    dir, err := os.Getwd()
    if err != nil {
        log.Println(err)
    } else {
        log.Println(dir)
    }
}

$ pwd
/home/meet/code/techstructive-blog

$ go run main.go
2022/10/01 19:19:09 /home/meet/code/techstructive-blog

So, as we can see the Getwd the function returns an absolute path to the current working directory which will be the path from which you will be executing/running the script file.

Get the path to the home directory

We can even get the home directory path like the /home followed by the user name on Linux and the User Profile with the name for Windows. The UserHomeDir() , returns the home directory for the user from which the file is being executed. The return value is simply an string just like the Getwd function.

package main

import(
    "os"
    "log"

)

func main() {
    dir, err := os.UserHomeDir()
    if err != nil {
        log.Println(err)
    } else {
        log.Println(dir)
    }
}

$ echo $HOME
/home/meet/

$ go run main.go
2022/10/01 19:35:50 /home/meet

So, as expected, the UserHomeDir function returns the path string to the home directory of the user.

Get path from a file name string

Let's say, we give in a filename and we want the absolute path of it. The path/filepath package provides the Abs function that does exactly that. The function returns a path string of the parameter parsed as a string to a directory or a file name. The function might as well return an error as the file path might not existing or the file might have got deleted, so we'll have to call the function with the ok, error syntax.

package main

import(
    "path/filepath"
    "log"
)

func main() {

    file_name := "default.md"
    log.Println(file_name)
    dir, err := filepath.Abs(file_name)
    if err != nil {
        log.Println(err)
    } else {
        log.Println(dir)
    }
}

$ go run main.go

2022/10/01 19:52:23 default.md
2022/10/01 19:52:23 /home/meet/code/techstructive-blog/default.md

As we can see the file default.md was parsed in the Abs() function and it returned the absolute path of the file.

Get Parent Directory from a Path

We can get the parent directory for a given path, if the path is to a file, we return the absolute path to the parent directory of the file, or if the path is to a folder, we return the folder's parent directory.

package main

import(
    "path/filepath"
    "log"
)

func main() {
    file_name := "drafts/default.md"
    //file_name := "drafts/"
    path, err := filepath.Abs(file_name)
    if err != nil {
        log.Println(err)
    } else {
        //log.Println(path)
        log.Println(filepath.Dir(path))
    }
}

$ go run main.go
2022/10/01 19:58:45 /home/meet/code/techstructive-blog/drafts

$ go run main.go
2022/10/01 19:58:45 /home/meet/code/techstructive-blog

As we can see when we parse in a file path i.e. drafts/default.md , the Dir the method returns a path to the parent folder, and even if we parse the directory path i.e. drafts/ , the method returns the parent of that directory.

Get the last file/folder for a given Absolute Path

Golang also provides a way to get the file/directory name from a path string using the Base function provided in the path/filepath package.

file_name := "default.md"
dir, err := filepath.Abs(file_name)

if err != nil {
    log.Println(err)
} else {
    log.Println(dir)
    log.Println(filepath.Base(dir))
}

$ go run main.go

2022/10/01 19:58:45 /home/meet/code/techstructive-blog/drafts/default.md
2022/10/01 20:19:28 default.md

So, the function Base will return the last element in the path, it can be a file or a directory, just returns the name before the last \ . In the above example, we start with a filename default.md but set the dir as the absolute path to that file and again grab the file name using the Base function.

Fetching details from a Path

We can even use utility functions for dealing with paths in golang like for checking if a file or path exists, if a path is a file or a directory, grabbing file name and extension, etc. The path/filepath and the os the package helps with working with these kinds of operations.

Check if a path exists

We can use the os.Stat function along with the os.IsNotExist for finding if a path is existing or not. The Stat function returns a FileInfo object or an error. The FileInfo object will have methods such as Name() , IsDir() , Size() , etc. If we get an error, inside the Stat method, the error will probably arise if the path does not exist, so inside the os package, we also have the IsNotExist() method, that returns a boolean value. The method returns true if the parsed error indicates that the path doesn't exist and false if it exists.

package main

import(
    "path/filepath"
    "log"
    "os"
)

func main() {

    file_name := "drafts/default.md"
    path, err := filepath.Abs(file_name)
    if err != nil {
        log.Println(err)
    } else {
        if _, err := os.Stat(path); os.IsNotExist(err) {
            log.Println("No, " + path + " does not exists")
        } else {
            log.Println("Yes, " + path + " exists")
        }
    }
}

$ go run main.go

2022/10/01 20:51:31 Yes, /home/meet/code/techstructive-blog/drafts/default.md exists

So, from the above example, the program will log if the path is present in the system or not. The error is parsed from the Stat method to the IsNotExist method for logging relevant messages. Since the directory exists, we get the path exists log.

Check if a path is a file or directory

The FileInfo object returned from the Stat the method provides a few methods such as IsDir() that can be used for detecting if a given path is a directory or not. The function simply returns a boolean value if the provided path points to a directory or not. Since we have to parse the path to the IsDir() function, we convert the file string into a path using the Abs method and then check if the path actually exist with the Stat() method.

package main

import(
    "path/filepath"
    "log"
    "os"
)

func main() {

    file_name := "drafts/default.md"
    //file_name := "drafts/"
    path, err := filepath.Abs(file_name)
    if err != nil {
            log.Println(err)
    } else {
        if t, err := os.Stat(path); os.IsNotExist(err) {
            log.Fatal("No, " + path + " does not exists")
        } else {
            log.Println(path)
            log.Println(t.IsDir())
        }
    }
}

$ go run main.go
2022/10/01 20:55:20 /home/meet/code/techstructive-blog/drafts/default.md
2022/10/01 20:55:20 false

$ go run main.go
2022/10/01 20:55:20 /home/meet/code/techstructive-blog/drafts/
2022/10/01 20:55:20 true

So, by running the program for a file and a directory, we can see it returns true if the path is a directory and false if the provided path is a file. In the above example, since the drafts/defaults.md is a file, it returned false , and for the next example, when we set the path drafts/ it returns true as the path provided is a directory.

Get File Extension from path

By using the path package, the extension of a given path can be fetched. The Ext method can be used for getting the extension of the provided path string, it doesn't matter if the provided path is exists or not, is absolute or relative, it just returns the text after the last . in the string. But if we are working with real systems it is good practice to check if the file or path actually exists.

package main

import(
    "path/filepath"
    "log"
    "path"
)

func main() {

    file_name := "default.md"
    dir, err := filepath.Abs(file_name)
    if err != nil {
        log.Println(err)
    } else {
        file_ext := path.Ext(dir)
        log.Println(file_ext)
    }
}

$ go run main.go
2022/10/01 21:03:23 .md

The above example demonstrates how we can get the extension of a file using the Ext() method in the path package. Given the string path as default.md , the function returned .md which is indeed the extension of the provided file.

Get Filename from path

We can even get the file name from a path in golang using the TrimSuffix method in the strings package. The TrimSuffix method trim the string from the provided suffix, like if we have a string helloworld and we provide the suffix as world , the TrimSuffix the method will return the string hello , it will remove the suffix string from the end of the string.

package main

import(
    "path/filepath"
    "log"
    "path"
    "strings"
)

func main() {

    file_name := "default.md"
    dir, err := filepath.Abs(file_name)
    if err != nil {
        log.Println(err)
    } else {
        file_ext := path.Ext(dir)
        log.Println(file_ext)
        log.Println(strings.TrimSuffix(dir, file_ext))
        log.Println(strings.TrimSuffix(file_name, file_ext))
        //log.Println(strings.TrimSuffix(dir, path.Ext(dir)))
        //log.Println(strings.TrimSuffix(file_name, path.Ext(dir)))
    }
}

$ go run main.go

2022/10/01 21:09:39 .md
2022/10/01 21:09:39 /home/meet/code/techstructive-blog/default
2022/10/01 21:09:39 default

We can use the TrimSuffix method to remove the extension as the suffix and it returns the path which we get as the file name. The TrimSuffix method returns the path after removing the extension from the path.

List Files and Directories in Path

In golang, we can use the io and the path/filepath packages to iterate over the file paths. Suppose, we want to list out all the files or directories in a given path, we can use certain functions such as Walk , WalkDir to iterate over a path string.

There are certain types of iterations we can perform based on the constraints we might have, like iterating over only files, or directories, not including nested directories, etc. We'll explore the basic iterations and explain how we fine-tune the iteration based on the constraints.

List only the files in the Path

The first example, we can take is to simply list out only the files in the current path directory, we don't want to list out the file in nested directories. So, it will be like a simple ls command in Linux. Let's see how we can list out the files in the given path.

We can even use path/filepath package to iterate over a given path and list out the directories and files in it. The filepath.Walk or the WalkDir method is quite useful for this kind of operation, the function takes in a path string and a WalkFunc or the WalkDirFunc Function, the walk function are simply used for walking of a path string. Both functions take two parameters, the first being the string which will be the file system path where we want to iterate or walk, and the next parameter is the function either WalkFunc or WalkDirFun respectively. Both functions are similar but a subtle difference in the type of parameter both take in.

WalkDir Function

The WalkDir function takes in the parameters such as a string of the file path, the fs.DirEntry object and the error if any. The function returns an error if there arises any. We have to call the function with the parameters of a string and a function object which will be of type type WalkDirFunc func(path string, d DirEntry, err error) error .

We can even use Walk the function to iterate over the given path.

Walk Function

The Walk function takes in the parameters such as a string of the file path, the fs.FileInfo object and the error if any. The function returns an error if there arises any. We have to call the function with the parameters of a string and a function object which will be of type type WalkFunc func(path string, info fs.FileInfo, err error) error .

It might be a user preference to select one of the functions for iterating over the file system, but the documentation says, the Walk function is a little bit inefficient compared to the WalkDir function. But if performance is not an issue, you can use either of those based on which type of file system object you are currently working with.

package main

import(
    "path/filepath"
    "log"
    "io/fs"
)

func main() {

    var files []string
    dir_path := "."
    err := filepath.WalkDir(dir_path, func(path string, info fs.DirEntry, err error) error {
        dir_name := filepath.Base(dir_path)
        if info.IsDir() == true && info.Name() != dir_name{
            return filepath.SkipDir
        } else {
            files = append(files, path)
            return nil
        }
    })

    if err != nil {
        panic(err)
    }
    for _, file:= range files {
        log.Println(file)
    }
}

$ go run walk.go

2022/10/02 12:07:17 .
2022/10/02 12:07:17 .dockerignore
2022/10/02 12:07:17 .gitignore
2022/10/02 12:07:17 CNAME
2022/10/02 12:07:17 Dockerfile
2022/10/02 12:07:17 README.md
2022/10/02 12:07:17 markata.toml
2022/10/02 12:07:17 requirements.txt
2022/10/02 12:07:17 textual.log

In the above example, we have used the WalkDir method for iterating over the file system, the directory is set as . indicating the current directory. We parse the first paramter as the string to the WalkDir function, the next parameter is a function so we can either create it separately or just define an anonymous function . It becomes a lot easier to write an anonymous function rather than writing the function separately.

So, we have created the dir_name variable which parses the dir_path from the parameter to the function and gets the name of the directory or file. We can then fine-tune the requirements of the iteration of the directory, i.e. make checks if the path is a file or a directory and if we want to exclude any specific files with certain extensions or directories with a certain name, etc. In this example, we have added a check if the path is a directory(using info.IsDir() ) and if the directory name is not the same as the parsed path(i.e. exclude the nested directories) we skip these types of directories (using filepath.SkipDir ). So we only look for the files in the current directory or the directory which we provided in the paramter as dir_path . We append those paths into the files array using the append method. Finally, we check for errors in the parsed parameter while iterating over the file system and panic out of the function. We can then simply iterate over the files slice and print or perform operations as required.

All the files in the Path (inside directories)

We can also list all the files within the folders provided in the path string by removing the directory name check. We will only append the file type to the file slice rather than appending all the directories.

package main

import(
    "path/filepath"
    "log"
    "io/fs"
)

func main() {

    var files []string
    root := "static/"
    err := filepath.WalkDir(root, func(path string, info fs.DirEntry, err error) error {
        if info.IsDir() {
            return nil
        } else {
            files = append(files, path)
            return nil
        }
    })

    if err != nil {
        panic(err)
    }

    for _, file:= range files {
        log.Println(file)
    }
}

$ go run walk.go

2022/10/02 12:08:22 static/404.html
2022/10/02 12:08:22 static/CNAME
2022/10/02 12:08:22 static/index.html
2022/10/02 12:08:22 static/main.css
2022/10/02 12:08:22 static/projects/index.html
2022/10/02 12:08:22 static/social-icons.svg
2022/10/02 12:08:22 static/tbicon.png

As we can see the iteration resulted in printing all the files in the given path including the files in the subdirectories. The static directory had the projects directory as a subdirectory in the path, hence we are listing the files in that directory as well.

Recursive directories in the Path

We can also append the directory names as well as file names by completely removing the info.IsDir() check and add the printing out of the relevant information as dir and files depending on the type. We can also maintain different lists or slices for directory and file and append them accordingly.

package main

import(
    "path/filepath"
    "log"
    "io/fs"
func main() {

    var files []string
    root := "static/"
    err := filepath.WalkDir(root, func(path string, info fs.DirEntry, err error) error {
        files = append(files, path)
        var f string
        if info.IsDir() {
            f = "Directory"
        } else {
            f = "File"
        }
        log.Printf("%s Name: %s\n", f, info.Name())
        return nil
    })

    if err != nil {
        panic(err)
    }

    for _, file:= range files {
        log.Println(file)
    }
}

$ go run walk.go

2022/10/02 12:09:48 Directory Name: static
2022/10/02 12:09:48 File Name: 404.html
2022/10/02 12:09:48 File Name: main.css
2022/10/02 12:09:48 Directory Name: projects
2022/10/02 12:09:48 File Name: index.html
2022/10/02 12:09:48 File Name: social-icons.svg
2022/10/02 12:09:48 File Name: tbicon.png

2022/10/02 12:09:48 static/
2022/10/02 12:09:48 static/404.html
2022/10/02 12:09:48 static/index.html
2022/10/02 12:09:48 static/main.css
2022/10/02 12:09:48 static/projects
2022/10/02 12:09:48 static/projects/index.html
2022/10/02 12:09:48 static/social-icons.svg
2022/10/02 12:09:48 static/tbicon.png

We can see that the directories and files getting logged which are present in the given path. In the output above, the projects the directory is getting walked along with the files present inside the directory. This is how we can use the Walk method to iterate over directories in a file system.

All the folders in the Path (only directories)

If we want to print only the directories, we can again add checks in the funciton body, we can simply append the path name when the path returns true on IsDir function call.

package main

import(
    "path/filepath"
    "log"
    "io/fs"
)

func main() {

    var folders []string
    root := "static/"
    err := filepath.WalkDir(root, func(path string, info fs.DirEntry, err error) error {
        dir_name := filepath.Base(root)
        if info.IsDir() {
            folders = append(folders, info.Name())
            return nil
        } else if info.IsDir() && dir_name != info.Name() {
            return filepath.SkipDir
        }
        return nil
    })

    if err != nil {
        panic(err)
    }

    for _, folder := range folders{
        log.Println(folder)
    }
}

$ go run walk.go

2022/10/02 12:13:25 static
2022/10/02 12:13:25 projects

Here, we can see it lists all the folder names present in the given path, it will log all the nested directories as well. In the above example, the static/ path in the local system had a projects directory and hence it prints the same, but that can be till the final depth of the file system.

For all the examples on the Walk functions, you can check out the links on the GitHub repository:

Relative or Absolute Paths

We have been using absolute paths in the above examples, but while navigating from one directory to other, we heavily make use of relative paths as they make it easier to move around.

Check if a path is Absolute

We can check if a path is absolute using the IsAbs function, the function takes in a path string as a parameter and returns a boolean value. It returns true if the provided path is absolute else it returns false .

Check if a path is Absolute

package main

import (
    "log"
    "os"
    "path/filepath"
)

func main() {

    dir, err := os.Getwd()
    if err != nil {
        panic(err)
    }
    log.Println(dir)
    log.Println(filepath.IsAbs(dir))

    dir = "../math"
    log.Println(dir)
    log.Println(filepath.IsAbs(dir))
}

$ go run rel_abs.go                                                                                                            
2022/10/02 14:38:44 /home/meet/code/techstructive-blog
2022/10/02 14:38:44 true
2022/10/02 14:38:44 ../math
2022/10/02 14:38:44 false

In the above example, we can see that when we parse ../math indicating there's a /math directory, before the current directory(parent directory) we get false .

But when we parse the path obtained from Getwd() function call or a path which is located from the root path will get the return value as true .

Get the relative path from base to target path

Let's say we are in a certain directory /a/b/c/ , we want to move into /a/c/d/ , we will have to move back two times and then move into c followed by the d directory. The relative path from /a/b/c/ to /a/c/d/ can be described as ../../c/d/ . We have a function in golang that does the same, basically creating a relative path from the base directory path to a target path. The function is provided in the path/filepath package as Rel , the function takes in two parameters, both as a string representing paths. The first is the base path(like you are in) and the second is the target path (as the target to reach). The function returns the string representation of the absolute path from the base to the target directory.

package main

import (
    "log"
    "os"
    "path/filepath"
)

func main() {

    dir, err := os.Getwd()
    if err != nil {
        panic(err)
    }

    dir, err = filepath.Abs("plugins/")
    s, err := filepath.Abs("static/projects/")
    if err != nil {
        log.Println(err)
    }

    log.Println(s)
    log.Println(dir)
    log.Println(filepath.Rel(s, dir))
}

$ go run rel_abs.go

2022/10/02 12:26:09 /home/meet/code/techstructive-blog/static/projects
2022/10/02 12:26:09 /home/meet/code/techstructive-blog/plugins
2022/10/02 12:26:09 ../../plugins <nil>

We can see that the relative path from the two directories is given as the return string from the Rel function.

Join paths

The Join method provided in the filepath package, is used for combining n number of path strings as one path. It separates the file paths with the operating system-specific separator like / for Linux and \ for windows.

package main

import (
    "log"
    "path/filepath"
)

func main() {

    dir, err := filepath.Abs("operators/arithmetic/")
    if err != nil {
        log.Println(err)
    }

    log.Println(filepath.Join("golang", "files"))
    log.Println(filepath.Join(dir, "/files", "//read"))
}

$ go run rel_abs.go

2022/10/02 12:30:37 golang/files
2022/10/02 12:30:37 /home/meet/code/techstructive-blog/operators/arithmetic/files/read

In the above example, we can see that it parses the path accurately and ignore any extra separators in the string path.

That's it from this part. Reference for all the code examples and commands can be found in the 100 days of Golang GitHub repository.

Conclusion

So, from the following post, we were able to explore the path package along with a few functions io as well as os package. By using various methods and type objects, we were able to perform operations and work with the file paths. By using functions to iterate over file systems, checking for absolute paths, checking for the existence of paths, etc, the fundamentals of path handling in golang were explored.

Thank you for reading, if you have any queries, feedback, or questions, you drop them below on the blog as a github discussion , or you can ping me on my social handles as well. Happy Coding :)

Golang: Math Package

Golang: Mutable and Immutable Data Types