Creations tend to inherit its creators' traits.

Soft computing

Soft computing is an umbrella term for a collection of computational methods designed to handle uncertainty, partial truth, approximation, and imprecise data. Unlike traditional 'hard' computing, which relies on strict binary logic and deterministic algorithms, soft computing embraces flexibility and adaptability to model complex, real-world problems where exact solutions are impractical or impossible.

Soft computing matters because it bridges the gap between rigid computational models and the messy, unpredictable nature of real-world problems, enabling robust solutions in domains like AI, robotics, and data science.

Bayes theorem

Bayes' theorem is a fundamental principle in probability theory that describes how to update the probabilities of hypotheses based on new evidence. It provides a mathematical framework for revising prior beliefs (initial assumptions) in light of observed data. The theorem is expressed mathematically as:

• Posterior probability (P[H ∣ e]): Updated probability of hypothesis H being true after considering evidence e.

• Likelihood (P[e ∣ H]): Probability of observing evidence e if hypothesis H is true.

• Prior probability (P[H]): Initial probability of hypothesis H being true before considering evidence e.

• Marginal likelihood (P[e]): Total probability of observing evidence e across all possible hypotheses.

For example, to determine the rational belief that a burglary has occurred given the alarm activation, we apply Bayes' theorem. The full story tells us that:

• Sensitivity (true positive rate): P(Alarm∣Burglary) = 95% = 0.95

• False alarm rate: P(Alarm∣No Burglary) = 1% = 0.01.

• Prior burglary probability: P(Burglary) = 1/10000 = 0.0001.

With Bayes' theorem, the total probability of the alarm activating is 0.010094.

Even though the alarm is 95% sensitive, the probability of an actual burglary given the alarm is only ~0.94%. This counterintuitive result arises because burglaries are rare (1 in 10,000 nights) and 1% of nights have non-burglary triggers (e.g., thunderstorms, pets).

A Naïve Bayes classifier employs Bayes' Rule to determine the probability of a classification outcome given multiple pieces of evidence, operating under the simplifying assumption that each piece of evidence is independent and equally influential. Mathematically, this is represented as P(C = c | x) = P(C = c) * P(x | C = c) / P(x), where its components refer to:

• Posterior probability (P(C = c | x)): Probability of class c given the observed features x. In an example of spam detection, this is the probability that an email is spam (C = 'spam') given the presence of words x like 'free' or 'offer'.

• Prior probability (P(C = c)): Probability of class c occurring without considering any features. Continuing the spam example, if 20% of emails in a dataset are spam, then P(C = 'spam') = 0.2.

• Likelihood (P(x | C = c)): Probability of observing the feature vector x given class c. Continuing the spam example, P('free' | 'spam') is the probability of seeing the word 'free' in spam emails.

• Marginal probability (P(x)): Overall probability of observing the feature vector x across all classes. Acts as a normalizing constant to ensure probabilities sum to 1. Also called evidence.

Completing the spam example, P(x) = P(x ∣ 'spam') * P('spam') + P(x ∣ 'not spam') * P('not spam').

How assumptions make computation easier

Though the Naïve Bayes classifier's simplified assumptions of computation from above rarely hold true in reality, it often performs surprisingly well. For instance, Suleiman et al. classified Android applications j as either benign or suspicious {0, 1} based on a feature vector of the app. Classification is then determined by selecting the class with highest estimated probability (see formula below). Individual conditional probabilities P(R_i =r_i | C = c) are typically estimated from the frequency of feature values within a sample of malware and non-malware applications.

To bring up a complete project example, IBM provides a Naïve Bayes formulation for spam detection based on the presence of specific words within an email. The predicted class label (spam or not spam) is determined by finding class y in the set of possible classes Y that maximizes the product of the prior probability of the class P(y) and likelihood of each word word_i (where i belongs to set of words I) given that class, P(word_i | y):

To simplify computation and avoid underflow, the log operator can be applied:

Fuzzy set theory

Fuzzy set theory is an extension of classical set theory that allows elements to have degrees of membership in a set, represented by a value between 0 and 1. This framework addresses situations where boundaries between categories are vague or gradual, rather than strictly binary. Key concepts about it include:

• Membership degree (μ): Element’s membership in a set is quantified by μ ∈ [0, 1]. For example, a person of height 183 cm might have μ = 0.6 in the fuzzy set "tall people," indicating partial membership.

• Fuzzy logic operations: Reason with AND intersection (e.g., μ_(A∩B) = min(μ_A, μ_B)), OR union (e.g., μ_(A∩B) = max(μ_A , μ_B)), IMPL implication (e.g., "IF temperature is high, THEN fan speed is high") with fuzzy variables. Hence the name fuzzy logic.

• Membership function: Mathematical rule that maps input values (e.g., height) to membership degrees.

Unlike classical set theory, which has strict boundaries, fuzzy set theory is like having bins where the categories overlap. To bring up a lecture example, the influenza vaccination guideline is not a sharp 'yes' or 'no' at exactly age 65. Fuzzy set theory allows the 'urgency of recommendation' to be a fuzzy set in the context of:

• 65-69 years old: Low membership degree.

• 70-84 years old: Strong membership.

• 85 years old: Very strong membership.

Membership function is the rule that determines this degree of 'urgency' based on age, creating a gradual transition rather than an abrupt change at a fixed threshold. This 'fuzziness' allows a clinical decision support system to understand that while 64 is close to 65, the recommendation is not as strong as it is for someone significantly older.

Fuzzy set theory: linguistic variables

Fuzzy membership is often employed to translate vague linguistic terms into quantifiable values. This process of 'fuzzifying' knowledge or expert opinions, which finite state transducers (FSTs) can model, inherently involves a degree of imprecision. Notably, there is no universally accepted dominant theory for this type of fuzzification, and fuzzy membership functions, in general, often represent best available estimates.

For example, terms like 'always' or 'urgent' might be mapped to a membership degree μ of 1, while 'very often' or 'strongly' could correspond to μ = 0.8, 'frequently' to μ = 0.6, 'seldom' or 'weakly' to μ = 0.3, and 'never' or 'irrelevant' to μ = 0. The assignment for 'unknown' can be subjective, perhaps μ = 0.4, reflecting an optimism level.

Fuzzy set theory: aggregation operators

Often, we need to combine multiple factors. Consider the desire to marry a 'rich and beautiful' person, with criteria μ1 representing richness and μ2 representing beauty. If we have candidates like Pat (richness: 0.3, beauty: 0.3) and Alex (richness: 0.8, beauty: 0.3), fuzzy set theory offers various aggregation operators:

• Classic AND: Uses min(μ_1, μ_2, …, μ_n). With this, Pat and Alex would both have a combined score of 0.3.

• Classic OR: Uses max(μ_1, μ_2, …, μ_n). Here, Chris (richness: 0.8, beauty: 0.8) would have the same score as Alex.

The Yager operator provides an alternative, moving from a sum-based approach towards a classic OR-like behavior. For example, with p = 2, w_1 ​= 0.4 (weight for richness), and w2​ = 0.6 (weight for beauty):

• Pat's score: min(1, [(0.4 * 0.3)^2 + (0.6 * 0.3)^2]^[1/2]) ≈ 0.216

• Alex's score: min(1, [(0.4 * 0.8)^2 + (0.6 * 0.3)^2]1/2) ≈ 0.367

Fuzzy logic

Fuzzy logic extends classical logic by allowing variables to have degrees of truth between 0 and 1 (completely false and completely true). This is particularly useful for modeling real-world scenarios where boundaries are ambiguous or subjective. In a rule like "IF x AND y THEN z," both antecedents (x and y) and the implication itself (THEN, represented by a rule strength) can be fuzzy.

For instance, if membership of x (μx) is 0.8, membership of y (μy) is 0.7, and the AND function is defined as the maximum, with a rule strength of 0.7, then the membership of the conclusion z (μz) after the rule fires would be max(0.8, 0.7) * 0.7 = 0.56. Conclusions can then be ranked based on their membership values, a concept similar to certainty factors used in the MYCIN expert system.

Fuzzy logic has achieved significant success in automated control applications. Examples include autopilots for both manned and unmanned aircraft and ships, automatic transmissions in vehicles, and anti-lock braking systems (ABS). It is also prevalent in consumer electronics like vacuum cleaners, washing machines, and refrigerators, where it manages parameters such as temperature, power consumption, and humidity. These applications often employ a set of triangular membership functions to characterize different situations.

Fuzzy control

Fuzzy logic is a valuable tool in control systems, allowing inputs to be based not only on the error e(t), the difference between the desired and actual state, but also on the rate of change of that error Δe(t) = e(t) - e(t - 1). This enables more nuanced control actions.

For example, if a robot vacuum cleaner is below its target suction level but the dust level is decreasing, the control action to increase suction might be less aggressive than if the dust level were still high or increasing. 

Fuzzy control systems often employ a substantial rule base to map input conditions to appropriate output actions. The following table, adapted from Ciliz, illustrates a partial decision table for a hypothetical 4-sensor robot vacuum cleaner.