Machine Learning versus Rule-Based Systems

There is a famous saying that goes, “Learn the rules like a pro, so that you can break them like an artist.”

It’s certainly true in today’s world, where everyone is so fond of breaking rules. Let’s use a simple example: a child eager for cookies. You want to make sure that they don’t eat more than four cookies in a day. To ensure that, you’ve set a rule that they can take only four cookies from the box at a time or they’ll get in trouble. The kid tries to take five or six cookies, and you send them to time out; the next time they try taking three cookies, and you don’t pay much attention. It’s not too much for you, and not against your rule either.

Now, the kid starts to understand the rule, your level of attention to different cases, your threshold and the chances of being caught, and they try to find a way around it.

How do Rule-Based Systems Work?

This time, they try two cookies, then three, then two again until it’s too late for you to realize that the box is already empty.

Now, you have to change the rule. This is the fundamental flaw in a rule-based system; rules are predictable, can be circumvented, and require constant manual intervention for improvement.

But now think about how a machine would handle this case, understanding the pattern and behavior of the child, the total cookies in the box, the frequency of attempts, successes, failures, and threshold crosses, and the list goes on.

With every cookie-sneaking attempt, the rule changes and the machine behaves differently. Will that confuse the kid? Yes.

Will they still keep trying? Yes — it’s a kid, after all.

Will they succeed in getting past the machine after taking four cookies? Nope.

In the same way that the child figures out the rules and sneaks past you for cookies, fraudsters can learn to bypass security-based rules in an effort to defraud banks or other institutions. Security based on self-evolving, self-improving machine learning platforms can prevent fraudsters from circumnavigating your systems.

Why Should Businesses Avoid Rule-Based Systems?

In other words: Why use machine learning vs rule based?

For years, the world has been following the same logic, “If X, then do Y, else if A, then do B.”

Does this work? Yes — that’s the elementary thing we learn in any language. But let’s increase the complexity and put this to the test in the real world. Say you are a money lending company, and to understand the credibility and eligibility of a customer, one has to pass through the 1,000 rules you have defined using “if” conditions.

Now consider the following situations:

• You have to change one of the rules somewhere in the middle of the list. You have to go through a lot of them to find the right one and repeat this process each time a rule needs a change.
• You have to add another 1,000 rules. You start to add to the list, and after a while the list becomes too difficult to manage and has a lot of redundancies.
• If you have one rule out of place, it can cause a number of false positives, which can be a big loss to the business.
• The person who wrote the initial rules for you leaves, and you have to spend time and resources to catch up on the long list of rules.
• The rules are difficult to trace back in the event of any exceptions, and a person needs to know all the rules that are in place in order to do this.

Do you see the problems here? When every company wants to build scalable systems to handle terabytes of data, it’s not feasible to stick with rules-based decision systems.

Traditional Rule-Based System Versus Machine Learning

As the world of data expands, it’s time to look beyond binary outcomes by using a probabilistic approach rather than a deterministic one. For example, what happens if you ask your system a question about a customer’s loan repayment?

A deterministic system will put in all the factors as per the rules and tell you whether the person will repay the loan. A probabilistic system can give you wider insights into how much of the loan the person is likely to repay, or the probability of the person committing fraud and hanging you out to dry. The probabilistic system relies on the world of machine learning.

Instead of “based on the inputs you have given me, here is the result,” machine learning works on the concept of “based on historic outcomes in this situation, here is what we can say about the future.”

Machine learning takes a probabilistic approach by using historical data and outcomes. It considers not only the input but n other factors. Machine learning understands patterns and trends in historic data, and gives you the probability of different outcomes based on that.

Machines Need to Be Trained

The big question is, how does a machine become informed about historic events? That process is called training.

Machine learning works through models that rely on algorithms that can be set to be totally data-driven. Compared to rule-based algorithms, machine learning scales very well with data. The more data points an algorithm learns from, the better the features it learns, and the better its performance becomes. One of the key challenges here is the availability and quantity of labeled data.The unlabelled data can be creatively used in self supervised algorithms to pretrain a machine learning model. That can then be fine-tuned through supervised algorithms to further refine the model on ground truth labels.

In the long run, machine learning is better than rule-based systems because of the models that can adapt to changing trends and the flexibility to tweak the parameters involved. The less human involvement there is in defining rules ensures that the system is less predictable and hence harder to penetrate.

When it comes to optimizing outcome and accuracy of any system, consider ditching those unwieldy rule-based systems and implementing a machine learning platform.

Learn more about how Socure uses machine learning here.