K MEANS
The Premise
K-means clustering is a natural approach to this problem since the method “aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean.” This is a reasonable approach because songs in the same cluster are likely to be similar. We are interested in partitioning our songs into different clusters and build this model using KMeans from sklearn. The simplest recommendation system we built using this model involves clustering songs into 10 clusters (explanation below). Given a playlist, we will recommend songs from a pool of songs that are in the same clusters as the songs in the playlist. Using our 100 playlist dataset, we find that our algorithm will make recommendations from 2,417 songs, which is quite large and may suggest that the algorithm is too random. Thus we attempted to improve the model by adding additional segmentation, resulting in an improved model that used artists more prevalent in the cluster to recommend songs.
To test the model (through calculating average R-precision and NDCG), we filter our data into playlists with a minimum 50 songs so that the model can use at least 25 songs for training. The average playlist in this dataset contains 52 songs and thus we want to recommend a playlist with that many songs. We do this by training the model on 25 songs and making it produce 50 songs to test on 25 songs.
Model Performance
Using 10 clusters, this model gave an NDCG score of 0.052 and an R-precision score of 0.011.
Selecting number of clusters
We can optimize the hyperparameter n_clusters by building an elbow curve (see graph) plotting number of clusters and amount of distortion. Based on our visual analysis, around 8 clusters seems to be the optimal tradeoff between number of clusters and distortion. We chose 10 clusters in order to ensure that we have enough clusters to capture variation in the data.
EXAMPLE PLAYLIST RECOMMENDATIONS
STARTING PLAYLIST
'7am', 'All The Way Up', 'Benjamins', 'Black Beatles', 'Body', 'Call On Me - Ryan Riback Extended Remix', 'Capsize', 'Caroline', 'Chill Bill', 'Closer', 'DEVASTATED', "Don't Wanna Know (feat. Kendrick Lamar)", 'Fake Love', 'Flex (Ooh, Ooh, Ooh)', 'Gassed Up', 'Girls Like (feat. Zara Larsson)', 'How I Feel', 'Hurts So Good', 'I Like Tuh', "If It Ain't Love", 'Ignition - Remix', 'Influence', 'Insomnia', 'Juju On That Beat (TZ Anthem)', 'Just Girly Things', 'Keep It Mello (feat. Omar LinX)', 'Let Me Explain', 'Let Me Hold You (Turn Me On)', 'Litty (feat. Tory Lanez)', 'Love Me', 'Man Of The Year', 'Mercy', 'Midnight City', 'New Thang', 'No Limit', 'No Role Modelz', 'No Type', 'One Night (Remix) [feat. Fetty Wap, Don Lu & King Nell$]', 'Phone Down', 'Really Really', 'Right On Time', 'Saved (feat. E-40)', 'Side To Side', 'Starving', 'Swimming Pools (Drank) - Extended Version', 'The Next Episode', 'The Show Goes On', 'This Could Be Us', 'Tiimmy Turner', 'Truffle Butter', 'Uber Everywhere', 'Ultimate', 'Wavy (feat. Joe Moses)', "We Can't Stop", 'White Iverson', 'X (feat. Future)', 'YNO', 'oui'
PREDICTED SONGS
'A Little Too Not Over You', 'All My Friends', 'All Night (feat. Knox Fortune)', 'All We Know', 'Aloha Ke Akua', 'Battles', 'Beware', 'Bitch Better Have My Money', 'Booty Me Down (Contest Mix)', 'California Gurls - feat. Snoop Dogg', 'Candyman', 'Cigarette Daydreams', 'Come Alive (Dry Bones)', 'Dammit', 'Dance for You', 'Drops of Jupiter', 'Facts (Charlie Heat Version)', 'Fancy', 'Feeling Myself', 'First Time', 'Fumble', 'Golden', 'Here', 'Hurricane - Arty Remix', 'I Bet My Life - Lost Kings Remix', 'I Stay In Love', 'I Want (feat. 2 Chainz)', "I'm Just A Kid", 'Jungle', 'Maybe', 'Me, Myself & I', 'My Homies Still', 'My Own Little World', 'My Story', 'My Victory', 'On This Vibe', 'Once In a While', 'Pretty Flacko', 'Prodigal', 'Rack City', 'Raging', 'Say It', 'SideLine Watching (Hold Up)', 'Slipping Away', 'Smile Like You Mean It', 'Sober', 'Sorry', 'Sorry Not Sorry', 'Super Mario World', 'TESTIFY', 'The Great Escape', 'Turn The Music Up', 'Used to This', 'What Would You Do', 'Who Gon Stop Me', 'Who Says', 'Work Out', 'Work REMIX'
ACTUAL FULL PLAYLIST
'679 (feat. Remy Boyz)', 'A Milli', "Ain't My Fault", 'All Time Low', 'All We Know', 'Bang Bang', 'Beware', 'BlackOut', 'Broccoli (feat. Lil Yachty)', 'Bugatti', 'Campaign (feat. Future)', 'Champions', "Day 'N' Nite (nightmare)", 'Dirty Vibe - DJ Snake & Aazar Remix', "Don't Mind", 'Famous', 'Feeling Myself', 'First Of The Year (Equinox) - Equinox', 'For Free', 'Froze (feat. Lil Uzi Vert & Nicki Minaj)', 'Hot N*gga', 'I Just Wanna...', 'I Want (feat. 2 Chainz)', 'In the Name of Love', 'Jimmy Choo', 'Jordan Belfort', 'Let Me Love You', 'Light It Up (feat. Nyla & Fuse ODG) - Remix', 'Live This Nightmare - NGHTMRE Remix', 'Lockjaw', 'Look Alive - Remix', 'Money Longer', 'Moolah', 'My Shit', 'Never Be Like You', 'No Problem (feat. Lil Wayne & 2 Chainz)', 'OOOUUU', 'Oh Lord', 'Once In a While', 'Rack City', 'Red Nose', 'Say It', 'Sexual', 'SideLine Watching (Hold Up)', 'Sit Still, Look Pretty (feat. R!OT)', 'Sunday Best', 'Sunset Lover', 'Super Mario World', 'Take Ü There (feat. Kiesza)', 'The Half', 'Trust Nobody (feat. Selena Gomez & Tory Lanez)', 'Used to This', 'Wake Up', 'Wherever I Go', 'Who Gon Stop Me', 'Who Wants to Rock', 'in my miNd', 'wonderful'