Create a PLaylist of Local Upcoming Shows with the Spotify API
Please check out my GitHub Repo for all of the code from this project!
NOTE: I’m currently in the process of updating this project and creating a web app that will allow any Spotify user to create a playlist on their own account. Stay tuned for that write-up!
One day in the not too distant future, when the scare of COVID-19 is long gone, we will all be running out of our doors once again to enjoy the finer things in life–restaurants, movies, sports, hiking, and LIVE MUSIC !
It is easy these days to go on the internet and see what bands will be playing soon in your area. Many of the larger bands you will have likely heard of, but what about the smaller acts? There are few things more exhilarating than finding a new band you love to support, and smaller acts truly need that support. It does, however, take a little more leg work to explore these lesser known artists and see if they’d be a good fit for your Friday night. With the Spotify API and a little web scraping, like me, you too can create a custom playlist full of a handful of songs from artists that will be performing in your area in the coming week!
The site bandsintown.com (which is full of lesser known, smaller, independent, and up and coming artists ) automatically populates a short list of the upcoming artists in your area. This makes a perfect recipe for some web scraping. Using the beautiful soup library you can easily grab band names off of this web site and then use spotipy to access the Spotify API in order to create a playlist.
In my GitHub repo I have both a jupyter notebook with individual code cells (that I used as a test notebook) and then also a python file with proper functions (that I will put into production). For this blog I’ll simply run through the test notebook so I can highlight the outputs, but if you want to see how it all comes together then you should go ahead and take a look at the .py file.
Outline
Creating an App on Spotify for Developers
Authentication Flow
Storing Credentials
Scraping the Bandsintown Website with Beautiful Soup
Cleaning the Data to Get a List of Upcoming Artists
Searching Spotify for These Artists
Creating a Playlist and Adding Songs
Creating an App on Spotify for Developers
The first step in this project (or really any project that would use the Spotify API) would be to navigate over to the Spotify for Developers page, create an account, and register an app. They have awesome documentation and even a Web API Quick Start Guide, which is where I recommend you start.
On your main Dashboard page you’ll have the opportunity to create or edit an app.
You’ll be redirected to your apps main dashboard page after you accept the terms of service. Here you will find you CLIENT_ID and CLIENT_SECRET which you’ll need to access later, so you may want to copy these down. Don’t share these, especially when posting code on GitHub or anywhere else.
Authentication Flow
Spotify offers a handful of ways to authenticate with it’s API. If you want to explore the different methods you can read about the different flows here. For this project I’d like to eventually create a simple web app where any user can log in to Spotify and create a playlist on their own account, so I’ll be going with the Authorization Code, which requires a secret key and uses tokens for authentication.
Storing Credentials
I’ll safely store my CLIENT_ID, CLIENT_SECRET, and REDIRECT_URI variables in my .env file and then access that file with the dotenv and os libraries. The REDIRECT_URI is simply where you would like the user to be redirected to after they login to your app. If you are going to be the only user and just running this on your own machine then it really isn’t important. If you are planning on having multiple users or having a web app then you will probably want some thing like this, pointing at a port on a server:
"http://localhost:8889/callback"
I’ll also use spotipy to interact with the Spotify API, but it is pretty straightforward if you’d rather work with the API directly. Here’s what the code will look like for authentication. At the end of the authentication you’ll notice I grab the current user’s user_id which will come in handy to create the playlist later on.
Spotify has several scopes that you can choose to work with. Essentially they grant your app “need to know” privileges, so that when a user logins in through Spotify they can be more assured that you app will not be able to access too much information from them – only the information necessary to perform the main function of your app. I’ll be using the playlist-modify-public scope since I’d like to be able to create and add to a public playlist. You can read more about their different scopes here.
Scraping the Bandsintown Website with Beautiful Soup
The Bandsintown website actually has two different APIs, one for artists and one for events. These are really great tools for artists, but to use them you need to input an artist’s name and they will return various information and objects to you. For my purposes here I’d actually like to discover artists based off of my location. The beauty of simply using their website rather than their API is that their website will detect my location automatically and then populate the upcoming artists accordingly. You can also alter their url to incorporate today’s date and choose a date range of events. I’ll use the datetime library to create and format today’s date and then insert that into the url with f-string formatting. Then I’ll set the date filter in the url to search for this week’s upcoming events, like this:
&date_filter=This+Week
The requests library requires that I pass in headers into the headers parameter. This is just metadata containing information on your operating system and browser. Depending on if you are using a Mac or Windows you can look these up in the Developer tools in your respective browser. In actuality I’ll only need the User-Agent variable, which you can simply call as USERAGENT, since it’s stored as an environment variable. After I make a connection to the website you can reference the status_code to determine if the connection was successful.
Now that I’ve connected to the website and have had a response object returned to me, I can use the Beautiful Soup library to parse the html. Below you can see that on the website the events are listed on the bottom of the page. If you right click on the first event and select Inspect Element you can see that one way we can grab the artist’s names is by accessing the href attribute, which contains a url comprised of the artist’s name. Also notice how it automatically recognized my location.
There are many other urls contained in the html on the website. My method was to extract all of the urls with regex and then filter that list with a list comprehension. The event urls are unique to the other urls in the html as they end with an /e/, like this:
https://www.bandsintown.com/e/
Cleaning the Data to Get a List of Upcoming Artists
Now that I have a list of urls containing artist names, I can work toward cleaning them up. I notice that following the artist’s name in the url is always -at-some-venue. So I created another regex pattern to and used the groupmethod to select the left side of the string url at that point.
Next since there are no other hyphens in the string url beside what is surrounding the artists name, I can replace the hyphens with spaces to isolate the artist’s name from the string. You might be thinking this was an odd strategy, but originally I ran into the edge case where a band’s name started with a number, which broke my regex pattern and this is my work around.
Now I’ll use the space to split off the artists name. Do you notice anything?
There are so many duplicates, so to get unique artist name’s I’ll just use a set function and then return it to a list.
And there we have a list of unique band names courtesy of bandsintown.com, ready to be passed into the Spotify search bar.
Searching Spotify for These Artists
Since many of these artists are smaller local artists, there’s no real guarantee that they are even on Spotify at all. Or perhaps they are only featured on a larger artist’s track. Or maybe the spelling is off. So when searched in Spotify they may not return the exact artist I’m looking for. I will then use a special string matching library that allows me to set a threshold for acceptance. The fuzzywuzzy package makes matching strings a breeze!
Here I use the search endpoint to pass in each artist name from my list and return 5 tracks maximum. If the artist’s name matches at a 75 threshold then I will append these track_id’s to a list.
Create a Playlist and Adding Songs
Spotify has many endpoints available for use, each with their own scopes, inputs, and parameters. You can view them in Spotify’s awesome documentation here. Since I’ll be creating a public playlist, I’ve made sure to already specify the playlist-modify-public scope, and as you can see below I’ll also need the user_id, which I grabbed earlier during authentication.
Armed with a big list of track id’s ready to populate into a playlist, I’ll first check to see if there already exists a playlist with this name. If it already exists I’ll replace it’s tracks, or if it doesn’t exist I will create a playlist first and then add tracks.
And that’s it! There should now be a new playlist on my Spotify account populated with tracks from artists that will be playing in my area within the next week. Time to start discovering my new favorite bands!