I tried to learn some deep learning, and I got stuck. I was reading a book, and I could almost feel information bouncing off my brain. Literally bouncing off like a ball bouncing off a wall.
I wondered how to understand the content and assimilate knowledge if I can’t even remember it.
Something was wrong, but I could not understand what was wrong. Something had to be wrong. Since my childhood, I was trained to learn incorrectly. The education system in my country is based solely on rote memorization. I am aware of other techniques, but I still learn by repetition, even though I know it does not work.
I wanted to learn the topic and all its details. It is a great goal when you are about to become an expert. It is a terrible idea when you start something new.
Why did I make such a mistake? I assumed that all I need to do is learning the mathematical theory and writing code.
It makes you think that if you have a few years of experience writing business applications, you can effortlessly use Tensorflow. It is just code, isn’t it? It is a very short code. Most of the examples consist of less than 200 lines of code. So what was the problem?
I had a goal: understand neural networks and use them. Deep learning is just a combination of other skills, isn’t it?
It turns out people don’t learn by reading books. Isn’t it shocking? I thought I needed to spend more time learning math, but that was a terrible mistake. Math is a detail. Beginners are not ready to learn details.
Beginners should find a good tutorial and just follow instructions. When we are beginners, we should ignore the desire to understand the concept and focus on accomplishing a simple goal.
Why do you need to learn?
According to Michael Simmons, “Knowledge is the new money.” Learning is the best investment of your time. In my opinion, you cannot be a programmer if you don’t want to learn. It is the only skill that matters. That is why I disagree with people who say everyone can be a programmer. Most people don’t want to learn. They think learning is a good idea but are not willing to make any effort.
That already is a massive problem because “Not learning at least 5 hours per week is the smoking of the 21st century” (quote by Michael Simmons).
How people learn
You don’t need the details, so always start with the general concept. Details are useless if you can’t understand the principles. Also, you can always google the details, if you know what you are looking for.
So, let’s talk about the efficient way of learning but without the details. If you want to know them, you should read “Pragmatic thinking and learning” by Andy Hunt. Let’s focus on the concepts that are important when you got stuck and can’t continue learning.
Dreyfus model of skill acquisition
Maybe you heard about the Dreyfus model of skill acquisition. It describes the way from a novice to an expert. The model consists of five stages. Every stage is different. Every stage has its own rules and guidelines. At every stage people learn differently. If you try to learn in a way that is not effective at your current stage, you will only waste your time.
From the perspective of a person who learns a new skill, the most important information is the fact that Dreyfus model is applicable per skill. You may be a proficient or even an expert programmer, but when you learn something new (for example, deep learning) you start as a novice, and you must act like a novice.
In the beginning, you should ignore the details. Details are not important yet. Surprisingly, you should also ignore the “big picture.” In the beginning, we need something straightforward. Novice needs quick successes and a lot of rules. Preferably context-free rules which are always applicable.
How do children learn to walk? Do they listen to lectures? Do we show them a 3-hour long PowerPoint presentation about using legs? Do we tell them how muscles work even before they try to make the first step? Do they care about the details? We don’t teach them about adenosine triphosphate before they make the first step, don’t we?
We do it differently. Children observe adults and try to mimic their behavior. They learn by doing and experimentation. They fail and make mistakes. Adults should do that too.
Learning by doing
Have you heard of SMART goals? Such goals are a perfect way to learn new skills. As a novice, you need a specific goal, something achievable, something you can do quickly. You are not trying to achieve something vague like “understanding.” At the novice stage, all you want is finishing a task.
If you learn something related to programming, play with the code and have fun. Can you break it? What happens when you change parameters? What if you can’t finish the task? Do you know what went wrong? Can you split it into smaller goals? Is the problem similar to something you already know?
Did you know that we are supposed to have fun while learning? How!? Be curious and use the scientific method. You read something, you noticed something interesting or strange, and you have a question. You may ask: “I wonder what happens when I do X.” What should you do? Google it? Ask a question on Stackoverflow? I often see such questions on Stackoverflow. Many of them are wasted learning opportunities. The authors would learn way more if they tested the code themselves instead of asking what would happen when they ran it.
What should happen after you ask a question? You should think about a possible answer! That answer becomes your hypothesis. What is the next step? You test the hypothesis. It sounds like something difficult. How can you do that?
After you form a hypothesis, you make a prediction. The prediction is something you expect to see if the hypothesis is correct. Now, it is time to test the prediction. Is it true or not? Remember that you cannot prove the hypothesis is correct. All you can do is rejecting the hypothesis or concluding that you don’t have enough evidence to reject it (the hypothesis may be true, but it does not mean you proved it)
If you think that the scientific method is too complicated and takes too much time to finish, you may be right. It is not something you should do when you develop a new feature at work. In such a situation, you should ask questions and get the answers quickly.
Ask questions if you want to get the job done, do your own research if you want to learn.
One more remark, copy/pasting is not doing. You should write code, even if it means writing the same text you see in another window on the same screen. You should write it, make mistakes, find the errors, and fix them. Copy/pasting is useless when you want to learn something.
Teaching is not optional. If you want to understand the concept, you should teach someone else. It will force you to find a simpler explanation, metaphors, and examples.
If you don’t believe, think about talking to a “rubber duck.” It sounds like a weird debugging technique, but it really works! If you explain your problem to an inanimate object (the duck!) or another person, you start thinking about it differently. You don’t even need an answer or advice. The sole act of explaining is sufficient to solve the problem.
Feynman technique and First Principles
You should explain (or at least pretend to explain) to someone who does not know anything about the subject. You want to make the subject as easy to understand as possible. You may look for uncomplicated definitions, write a story, find some analogies or metaphors. The tools do not matter. The only important part is a simple explanation.
Yes, I know, Elon Musk disagrees with that. In my opinion, his advice is not for beginners. It works great when you are already know something about the subject. In the beginning, you can’t even recognize the “first principles.”
“It’s important to reason from First Principles rather than by analogy. So the normal way that we conduct our lives is that we reason by analogy. We are doing this because it’s like something else that was done. Or it’s like what other people are doing. (…) It’s mentally easier to reason by analogy, rather than from first principles. — Elon Musk
It is easy to read a blog post, think about a new idea for a few seconds, maybe even agree with it and do nothing. It is also easier to watch a lecture and do nothing. Does it mean that you have learned anything? You can memorize the content of a book. Does it mean anything? You can probably pass a test if you remember everything, but do you understand it? Can you use the knowledge in practice?
As Marty Lobdell said, “If it does not change your behavior, you have not learned it.” Change the way you learn.