While training a model, a few important mathematical tricks have to be employed. Normalization and regularization techiniques belong to them. They are very important to successfully train models, and they are often confusing and sometimes hard to understand the details as well. Additionally, learning rate scheduling is also very important while training models, especially large ones. Normalization vs Regularization To put it simple, normalization is applied on data. Before training a model, data should be normalized. There are a few irresistible benefits to do it. It stablizes the training process, speeds up model convergence significantly, and improve model’s generalization capability. ...
Built-in Data Structures and Algorithms in Python
It’s very important to choose the proper data structure and algorithm for various programming tasks. Even when we are using high-level programming language such as Python, it’s still crucial in terms of time and space complexity. In this post, I’ll try to summary all built-in data structures and algorithms in Python as a recap and reference. There are so many details which could be covered in this post, please refer to Python’s official documents. ...
Prompt Engineering Review
First Update on Oct. 2nd, 2025 How you prompt the fine-tuned Large Language Model (LLM), such as ChatGPT or Gemini, decides generally what responses you can get. These techniques or tricks are called Prompt Engineering, which is one of the very basic skills that everyone should know a bit in today’s AI era. It’s crucial for both our daily life while talking with chatGPT or Gemini and building LLM-based AI agents. ...
Backtracking Algorithm
Backtracking algorithm is an elegant way to realize nested loop using recursive to search the solution space for your problem, especially for those problems that the depth of nested loop could not be determined beforehand. In this post, I try to demonstrate how to understand and code backtracking algorithm step by step with lots of examples. Take Algorithm as Technology! What is Backtracking Algorithm? I would like to think of backtracking algorithm as a search method. Imaging you are in a forest and you need find a way out. There are multiple choices at each intersection, and you have no idea about which direction or path is right. So, you pick out one direction randomly or in any order you like and keep on going. If it is a dead end, you go back and choose another direction, otherwise you find a way out. During the whole process, there might be lots of “going back and choosing again” moments which is actually the spirit of backtracking. ...