jccheung.github.io

Renewable Energy Prediction

Prediction of Solar and Wind Renewable Energy using Deep Learning

Long Short Term Memory (LSTM) model was developed to monitor short term power output predictions from 4 hour ahead to 6 hours ahead. This research paper demonstrated a unique approach of utilizing nearby publicly available meteorological data to predict renewable energy power output that could potentially be used for wind and solar farm scheduling to prevent curtailment of clean energy.

This paper was written for completion of my Masters major research project under Toronto Metropolitan University and as a result the paper cannot be shared publicly without explicit permission from the University. It was written in Python with Pandas, Numpy, Tensorflow and other supporting libraries.

Questions to be Answered:

Can all of Ontario’s power demands be met with renewable energy?
What are the key features for determining renewable energy?
What would be the best algorithm choice to predict renewable energy?

Comparing electricity per month — *Data Directory style*

The dataset used was collected from the [Independent Electricity System Operator] and the [NASA Solar Satelitte dataset.]

The dataset made up of approximately 50 meteorological features total over a 4 year time period. Notable features such as wind speed, solar radiation, air temperature, dew point temperature and humidity all played a role in determining total solar and wind power generation capacity.

model_life contents — *Model lifecycle utilized in this project*

The data was then run through exploratory data analysis: simple correlation operations to determine the feature with the greatest correlation to wind power. The data was also cleaned and outlier values were either removed or replaced with average values.

Pearson_corr — *Pearson Correlation for Wind Power*

The Long Short Term Memory LSTM deep learning algorithm is trained on 3 years worth of data, with the 4th year being the validation/ testing year. This results in a 75% training, 25% testing split. The benefit of using LSTM networks over a traditional mathemetical model such as the ARIMA (Autoregression Integrated Moving Average), is that there doesnt need to be a processing step to remove trends and seasonality.

The Long Short Term Memory algorithm requires a number of timesteps, also known as an input window size. To determine how many past values of the series will be used for the current prediction. For example, if i wanted to predict 2 pm, should consider weather at 1 pm, 12 pm, and 11 am as my past values, meaning my window size is 3 hours.

Here we can see the output result for the solar farm, seperated by season. The algorithm is predicts the next 4 hours ahead with relative certainity. The wind farm is also predicted and both results are shown below.

Additional work would be to include more weather variables and try utilizing a CNN model to the algorithm before utilizing the LSTM.

Libraries:

Some of the tools required to make this project work:
Python - Python is a programming language that lets you work quickly and integrate systems more effectively.
Scikit-Learn - An open source Machine learning library in python used for data preprocessing, classification, regression, etc…
Pandas - Pandas is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language.
Jupyter Notebooks - web-based interactive development environment for notebooks, code, and data.

Climate Policy Vector Database

Vector Embeddings for climate policy legislation

Vector embeddings are essentially converting text or other forms of unstructured data such as audio and images into a high dimension matrix. This high dimensional space is known as an embedding, and can be used to increase the speed and accuracy of search results, as compared to semantic search. In a vector database, queries that have similarities are put together based on similarity, just like K nearest neighbours which makes them much faster to find compared to relational databases. With the ongoing development of technologies such as ChatGPT and other Large Language Models, there is an increasing need for an upgrade from the data base side to match the speed of the models. The enormous number of attributes and features that the model has to draw insights for is endless and take time and computing power to query searches. To solve this issue, using a specialized database that can provide a vector embedding and essentially group the data by semantic meaning, like a K nearest neighbour vector match to draw insights quickly and more accurately than a semantic database.

This project utilized the Climate policy radar database, which consisted of over 4000+ policies and papers that were written regarding climate.

As a researcher for example, almost always when you have a thesis or journal paper or novel idea, then you have to find background articles, papers that other people wrote that are similar but not exactly the same. This can be difficult to find as your idea is novel and assuming you are the first one to write a paper about it, it would not show up on a typical search engine, This is where vector embedded search comes in, if you were to search your paper details, you would be able to find the 10+ journal papers that have the relative closeness of your paper, and reference them much faster.

Objectives

Build a database based on the Climate Radar Policy
Create vector embeddings for the climate radar policy features
Write a python flask search application to query and display results back to users

Method

It was written in Python, using Docker with Weaviate and other supporting libraries.

A docker container was created for all its dependencies. The dataset used was collected from the [Climate Policy Radar]

The dataset made up of approximately 4000+ journals, laws and papers totaled over a 4 year time period. Notable features include, name, summary, category, sector, language, country of origin and outcome of the policy.


class_obj = {
        "class": "Article",
        "description": "Information regarding Climate Policy",
        "properties": [
            {
            "name": "Document_title",
            "dataType": ["string"],
            "description": "Document Title",
            },
            {
            "name": "Family_summary",
            "dataType": ["string"],
            "description": "Document Summary",
            },
            {
            "name": "Category",
            "dataType": ["string"],
            "description": "Document Category",
            },
            {
            "name": "Sectors",
            "dataType": ["string"],
            "description": "Document Sector Topics",
            },
            {
            "name": "Keywords",
            "dataType": ["string"],
            "description": "Document Multiple Keywords",
            },
            {
            "name": "Responses",
            "dataType": ["string"],
            "description": "Document Publish Response",
            },
            {
            "name": "Language",
            "dataType": ["string"],
            "description": "Written Language",
            },
            {
            "name": "Geography",
            "dataType": ["string"],
            "description": "Country of Origin",
            }
        ]
    }

Created Schema — *Schema that defines the data*

The data was then run through exploratory data analysis: simple data cleaning and text organizing then uploaded to the schema as well.

Flask application

The flask application is connected to the Weaviate Client.

Here the Weaviate nearText operator is used, to search for text that has the closest semantic meanings to the query that the user searched for. The response from the search query will include the closest objects in the climate policy class. The output response is then taken as the name, summary and details of the selected journal article.

Lastly the flask app front end and back end is created. Firstly the backend for POST requests from Weaviate is created and the search function to return the results from the client and render the page.

Populate Home page — *Populate homepage*

The index.html file

The search results.html file

The final website functionality

With relative ease of use, I have created a fully functional vector embedded search engine, for any database files.

Search result page — *Search results page*

This application is not limited to climate policy radar search engine data, the possibilities such as finding shoes that are similar to the ones you own, finding animals that look alike each other. The extensive possibilities for these technologoies with large language models such as chatGPT make it endless.

Libraries:

Some of the tools required to make this project work:
Python - Python is a programming language that lets you work quickly and integrate systems more effectively.
Weaviate - Weaviate is an open-source vector database. It allows you to store data objects and vector embeddings from your favorite ML-models, and scale seamlessly into billions of data objects.
Docker - Docker is a platform designed to help developers build, share, and run modern applications.

Robotic Painting Quality Control

How do you tell if two paintings are the same?

Museum curators spend millions of dollars every where to determine fake paintings from the real ones. Teaching curators to spot fake paintings and determine real stroke marks has always been a major challenge even with carbon dating. Utilizing computer vision to devise a method to compare painting stroke types or to perform some preliminary sweeps before a human will need to go in and look at the paintings, similar to applicant tracking systems and filtering resumes and cover letters before they are ever seen by a real person.

Some questions that needed to be answered were:

Are there methods to infer what paint brush was used for a specific paint stroke?
What is the best way to tell between two brush strokes with the exact same colour, and the exact same brush?

Starting with creating a pipeline to set up data base and data segmentation. Developing a dataset of paintings and individual paint strokes for comparison.

Comparing paintings — *How do you tell if a painting is the same?*

Multiple comparison scores are developed using OpenCV2. Colour matching, skeletoning, and texture detection are all run through the library to compare two paint strokes. Produced visual reports and comparison data to give an overall score on painting stroke types.

Skeletionization of stroke types using basic morphological operations

Skeletonization Example — *Example of skeletonization*

Performing thresholding, first converting an image to grayscale then into a binary image. Choosing a specific threshold, and pixels with values greater than the threshold will be white (255), and similar will be converted to black (0).

Thresholding Example — *Example of thresholding*

With thresholding only, there are parts of the image that might not be what we are looking for. In the example above notice that there are pixels between the cell images that we are not interested in. To remove these unwanted pixels, running basic morphological operations on the thresholded images.

Dilating and Erosion:

Dilating adds pixels to the boundaries of objects while erosion removes pixels on the object boundaries.

Dilation Example — *Example of Dilation*

Performing simple morphological operations after thresholding to get skeletonized images of paint strokes. Then running comparison of the skeletonized images by pixel values to compare paint strokes.

This was one of the many techniques used to compare images. Multiple additional processes done to compare the strokes provided a numerical percentage value of likeness. The full repository is private for company usage but this was simply an example of the operatons completed.

Potential future work is the utilization of GANs to create and mimic the paintings, then have the algorithm compare between the original paining the GAN created painting.

Libraries:

Some of the tools required to make this project work:
OpenCV2 - An open source computer vision and machine learning software library.
Scikit-Learn - An open source Machine learning library in python used for data preprocessing, classification, regression, etc…
Jupyter Notebooks - web-based interactive development environment for notebooks, code, and data.

Dog Breed Identification

Prediction and Detection of Different Dog Classes

Image classification has many excellent datasets for determining all types of classes, from humans, to airplanes to cats. VOC2012, a competition known as the Visual Object Classes Challenge to build a image database that has 20 clases ranging from persons, animals, vehicles and indoor furnitures. There is a classification for dogs, but not for dogs, so building upon this competition I wanted to train my own model to predict dog breeds.

This paper is built off of [R-CNN: Towards Real-Time Object Detection with Region Proposal Networks] written in PyTorch to determine the bounding boxes and prediction values for the dog breeds.

Some questions that needed to be answered were:

How many dog breeds are trained?
How is the data stored and organized, how are the features extracted.

The dataset used was collected from the [Stanford Dogs Dataset] with total of 120 categories, with annotations for class labels and bounding boxes in PASCAL VOC format. This meant the data needed to be in XML and JPG files like the following.

Comparing data folders — *Data Directory style*

For each image, there is an XML file that defines the size of the image, the class name as well as the bounding box location.

xml file contents — *XML file contents, notice the coordinates xmin, xmax and ymin, ymax showing the coordinates of the bounding boxes*

The data was then run through simple preprocessing morphological operations such as rotating, shifting as well as mirroring the images, to increase the size of the dataset and provide the algorithm with even more input data.

The machine learning algorithm is trained with the new dog breed classes and is compiled to learn and write the bounding boxes as an output. Postman is used as the testing and API call as it is upload to a localhost web server to try to receive POST calls from the server.

Here we can see the output result, the dog breed, as well as the prediction result in a tensor. The algorithm is 75% certain that this is an image of a Tibetan Mastiff, which is correct. This project was a demonstration of creating and replicated an algorithm from a journal article, as well as usage of an API platform for building a machine learning API in the future.

Additional work would be to include much more dog classes to the algorithm for training as well as work on the accuracy of the model to have higher certainty of the prediction. In addition, making the webpage look alittle better than a blank page with text. The github will continue to be updated to make these changes.

Libraries:

Some of the tools required to make this project work:
OpenCV2 - An open source computer vision and machine learning software library.
Scikit-Learn - An open source Machine learning library in python used for data preprocessing, classification, regression, etc…
Postman - Postman is an API platform for building and using APIs. Postman simplifies each step of the API lifecycle and streamlines collaboration
Jupyter Notebooks - web-based interactive development environment for notebooks, code, and data.