Difference between revisions of "Machine Learning"
From Living Building Science
| Line 4: | Line 4: | ||
We started off this semester by doing the LBS vip. As we were originally in Beesnap, our projects have been geared toward machine learning applications with the goal of integrating with the Beekeeper GO application. However since that is no longer the case, we have shifted our focus to more relevant issues for this VIP. With the biodiversity team's focus on birds and Dr. Weigel's bird sound data that she has, we decided that analyzing and classifying birds would be helpful. We initially reached our to the biodiversity team but they said that there would probably not be anything for us to collaborate on, so we decided to do the bird classifications on our own. Additionally, we are expanding the bee hive analysis to include varroa mite detection with the intention to monitor the health of the hives on campus. | We started off this semester by doing the LBS vip. As we were originally in Beesnap, our projects have been geared toward machine learning applications with the goal of integrating with the Beekeeper GO application. However since that is no longer the case, we have shifted our focus to more relevant issues for this VIP. With the biodiversity team's focus on birds and Dr. Weigel's bird sound data that she has, we decided that analyzing and classifying birds would be helpful. We initially reached our to the biodiversity team but they said that there would probably not be anything for us to collaborate on, so we decided to do the bird classifications on our own. Additionally, we are expanding the bee hive analysis to include varroa mite detection with the intention to monitor the health of the hives on campus. | ||
| + | === Bird Classification - Sound === | ||
| + | The goal of this project is to identify bird species based on audio with birdsongs. Dr. Weigel provided us with a list of bird species that are likely found near campus for us to focus on. She also referred us to two websites to collect bird sound data, but when we looked into it these were not adequate for a machine learning model since downloading was done one file at a time. We looked for our own sound data and found a large Kaggle dataset but it was a little too large for our needs (around 25 GB) so we decided to download the data on the DGX and extract only 10 species of birds to make exporting and downloading easy on our own machines. We used Dr. Weigel's list of birds to choose the 10 birds to focus on. | ||
| − | |||
| + | With image classification models, we use a Convolutional Neural Network (CNN) as our model to classify the images properly. However, with this classification, our data type is mp3 audio files. Therefore, to use a CNN, we mapped all of the audio files into spectrograms. | ||
| − | + | ||
| + | To map all of the audio files into spectrograms, we utilized the Python Librosa module. After research, we found that we cannot preprocess spectrograms like traditional image data. We instead tried to convert our spectrogram into a MFCC (mel-frequency cepstrum), which is supposed to remove the low frequency sounds as the bird songs are high frequency sounds. The formula we used to convert the spectrogram into a MFCC was that we took the log of the spectrogram data and then did a discrete cosine transform on it to get a MFCC. | ||
| + | |||
| + | |||
| + | After preprocessing the spectrogram to remove low frequencies, we ran the CNN model with on this preprocessed spectrogram (MFCC). However, after 50 epochs, it had a very low accuracy at 25%. This was very concerning as we could accurately the sound data with this model. We deduced that our preprocessing implementation could not be correct, so we ran the CNN with the original spectrogram images and after 50 epochs the accuracy was significantly higher at 71.8%. Though, we are not exactly sure how to accurately verify this accuracy by just looking at the spectrograms, because we do not know how to interpret them. Since we only focussed on 10 species of birds here, the next steps would be to gradually add in more and more bird species. I think that it is worth looking into removing the low frequency sounds again but maybe only a certain percentage. The end goal would be to use the raw data of bird sounds from campus and feed that into the model and get the desired output. | ||
| + | |||
| + | |||
| + | The code for this project can be found under the bird repository linked below in Spring 2021 Work Section. | ||
| + | |||
| + | |||
| + | Resource: https://towardsdatascience.com/sound-based-bird-classification-965d0ecacb2b | ||
== Spring 2021 Work == | == Spring 2021 Work == | ||
Revision as of 14:51, 29 April 2021
Welcome to the Machine Learning Team Page! One of our past goals last semester was to create a bee recognition and classification project to complement our finished flower recognition model and eventually integrate both of these models with the bee classification project of the previous ML team as well as the BeeKeeper GO app. Once successfully integrated, our models could potentially be used to enhance the experience of the app and make it more educational for users, as the classification/identification of flowers will allow users to learn while taking pictures. In addition, another goal of last semester was to use data analysis and computer vision to detect the occurrence of swarms in bee hives and the contributing factors/warning signs that will lead to a swarm occurring. Our current goals include improving and expanding our past projects as well as potentially creating machine learning and analysis tools to assist the other sub-teams in Living Building Science with their goals.
Contents
Spring 2021 Semester Overview
We started off this semester by doing the LBS vip. As we were originally in Beesnap, our projects have been geared toward machine learning applications with the goal of integrating with the Beekeeper GO application. However since that is no longer the case, we have shifted our focus to more relevant issues for this VIP. With the biodiversity team's focus on birds and Dr. Weigel's bird sound data that she has, we decided that analyzing and classifying birds would be helpful. We initially reached our to the biodiversity team but they said that there would probably not be anything for us to collaborate on, so we decided to do the bird classifications on our own. Additionally, we are expanding the bee hive analysis to include varroa mite detection with the intention to monitor the health of the hives on campus.
Bird Classification - Sound
The goal of this project is to identify bird species based on audio with birdsongs. Dr. Weigel provided us with a list of bird species that are likely found near campus for us to focus on. She also referred us to two websites to collect bird sound data, but when we looked into it these were not adequate for a machine learning model since downloading was done one file at a time. We looked for our own sound data and found a large Kaggle dataset but it was a little too large for our needs (around 25 GB) so we decided to download the data on the DGX and extract only 10 species of birds to make exporting and downloading easy on our own machines. We used Dr. Weigel's list of birds to choose the 10 birds to focus on.
With image classification models, we use a Convolutional Neural Network (CNN) as our model to classify the images properly. However, with this classification, our data type is mp3 audio files. Therefore, to use a CNN, we mapped all of the audio files into spectrograms.
To map all of the audio files into spectrograms, we utilized the Python Librosa module. After research, we found that we cannot preprocess spectrograms like traditional image data. We instead tried to convert our spectrogram into a MFCC (mel-frequency cepstrum), which is supposed to remove the low frequency sounds as the bird songs are high frequency sounds. The formula we used to convert the spectrogram into a MFCC was that we took the log of the spectrogram data and then did a discrete cosine transform on it to get a MFCC.
After preprocessing the spectrogram to remove low frequencies, we ran the CNN model with on this preprocessed spectrogram (MFCC). However, after 50 epochs, it had a very low accuracy at 25%. This was very concerning as we could accurately the sound data with this model. We deduced that our preprocessing implementation could not be correct, so we ran the CNN with the original spectrogram images and after 50 epochs the accuracy was significantly higher at 71.8%. Though, we are not exactly sure how to accurately verify this accuracy by just looking at the spectrograms, because we do not know how to interpret them. Since we only focussed on 10 species of birds here, the next steps would be to gradually add in more and more bird species. I think that it is worth looking into removing the low frequency sounds again but maybe only a certain percentage. The end goal would be to use the raw data of bird sounds from campus and feed that into the model and get the desired output.
The code for this project can be found under the bird repository linked below in Spring 2021 Work Section.
Resource: https://towardsdatascience.com/sound-based-bird-classification-965d0ecacb2b
Spring 2021 Work
Link to GitHub Repos: Swarms, Varroa, Birds
Past Semester Projects
Team Members
| Name | Major | Years Active |
|---|---|---|
| Sukhesh Nuthalapati | Computer Science | Spring 2020 - Present |
| Rishab Solanki | Computer Science | Spring 2020 - Present |
| Sneh Shah | Computer Science | Spring 2020 - Present |
| Daniel Tan | Computer Science | Fall 2020 - Present |
| Quynh Trinh | Computer Science | Fall 2020 - Present |
| Jonathan Wu | Computer Science | Spring 2020 - Present |