<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[ZhiZhi Gewu]]></title><description><![CDATA[For I am and always have been one of those natures who must be guided by reason, whatever the reason may be which upon reflection appears to me to be the best.

– Plato’s Crito]]></description><link>https://www.zhizhi-gewu.com</link><image><url>https://www.zhizhi-gewu.com/img/substack.png</url><title>ZhiZhi Gewu</title><link>https://www.zhizhi-gewu.com</link></image><generator>Substack</generator><lastBuildDate>Wed, 13 May 2026 12:33:48 GMT</lastBuildDate><atom:link href="https://www.zhizhi-gewu.com/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[KY John]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[zhizhigewu@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[zhizhigewu@substack.com]]></itunes:email><itunes:name><![CDATA[KY John]]></itunes:name></itunes:owner><itunes:author><![CDATA[KY John]]></itunes:author><googleplay:owner><![CDATA[zhizhigewu@substack.com]]></googleplay:owner><googleplay:email><![CDATA[zhizhigewu@substack.com]]></googleplay:email><googleplay:author><![CDATA[KY John]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[Interchange Fee and the Universal Acceptance of Cards]]></title><description><![CDATA[Payment is fundamentally a two-sided market. Cardholders want more merchants accepting their cards. Merchants want more cardholders. Increasing the number of merchants in the network comes with costs on the acquirer but also benefits the cardholders, and vice versa.]]></description><link>https://www.zhizhi-gewu.com/p/interchange-fee-and-the-universal</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/interchange-fee-and-the-universal</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 09 May 2026 08:00:45 GMT</pubDate><content:encoded><![CDATA[<p>Payment is fundamentally a two-sided market<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a>. Cardholders want more merchants accepting their cards. Merchants want more cardholders. Increasing the number of merchants in the network comes with costs on the acquirer<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-2" href="#footnote-2" target="_self">2</a> but also benefits the cardholders, and vice versa. In economic terms, it is called externalities.</p><p>We, therefore, cannot analyze the payments industry by looking at only one side. The total cost of payment equals the sum of the issuer&#8217;s cost<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-3" href="#footnote-3" target="_self">3</a>, the acquirer&#8217;s cost, and other frictions. Unfortunately, the cost structure is quite imbalanced for credit payment. The marginal cost of processing an additional transaction to issuers typically includes the cost of funds, cost of risk<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-4" href="#footnote-4" target="_self">4</a>, payment fraud, and customer incentives<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-5" href="#footnote-5" target="_self">5</a>. The marginal cost to acquirers is essentially zero once the infrastructure, such as a POS machine, is set up, ignoring transaction fees paid to the card scheme and issuers<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-6" href="#footnote-6" target="_self">6</a>. Increasing the number of cardholders benefits acquirers more because the acquirer keeps the MDR<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-7" href="#footnote-7" target="_self">7</a> from more transactions without doing anything extra.</p><p>Negotiations between issuers and acquirers on cost sharing are infeasible if universal acceptance is the goal. The number of bilateral agreements needed in the network, assuming only pure issuers and acquirers exist and their numbers are the same, is n&#178;. Card schemes, such as VISA and Mastercard, internalize externalities and reduce the transaction costs of negotiating bilateral agreements, thereby making universal card acceptance possible. In the court case between National Bancard Corporation (NaBANCO) and VISA in the 1980s<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-8" href="#footnote-8" target="_self">8</a>, the court concluded that &#8220;The IRF<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-9" href="#footnote-9" target="_self">9</a> is a mechanism by which VISA ensures the universality of its card, not a price fixing device to squeeze out entrepreneurs,&#8221; and &#8220;redistribution of revenues or costs is a must for the continued existence of the product.&#8221;</p><p>The court case also addressed the market in which VISA operates. NaBANCO believed there were three distinct markets: card issuing, merchant servicing, and interchange for receivables. VISA believed there was only a single market that included substitutes, such as cash, cheque, ATM cards, Amex, and Diners Club. The market definition dispute matters because interchange costs cannot be viewed solely as merchant costs. If customers can switch among substitutes, charging interchange fees shifts usage across payment instruments rather than simply lowering merchant cost.</p><p>In some markets, regulators cap interchange fees, including Spain, Australia, the United States, and the European Union. The International Center for Law &amp; Economics has written a paper on <em><a href="https://laweconcenter.org/resources/the-effects-of-price-controls-on-payment-card-interchange-fees-a-review-and-update/">The Effects of Price Controls on Payment Card Interchange Fees: A Review and Update</a></em>. The empirical evidence is mixed and institutionally contested. Critics of interchange caps argue that caps reduce issuer revenue, weaken rewards, and lead to incomplete merchant pass-through. Regulators such as the European Commission, however, argue that caps lowered merchant service charges and produced consumer benefits through lower prices or improved retail services.</p><p>In Southeast Asia, QR payment has become an important digital payment method. Unlike China, Central Banks in the Region act as intermediaries, providing national QR infrastructure and effectively setting the interchange fee to zero and capping the MDR to a very low level. The inclusion of credit issuers inevitably reduces. However, the Central Banks seem to optimize for payment-rail inclusion more than credit-rail inclusion and are determined to transition from a cash economy to a digital economy.</p><p>The framework, therefore, predicts which side has a structural advantage in each market. Where MDR and interchange are high, credit issuers can participate profitably in off-us transactions<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-10" href="#footnote-10" target="_self">10</a>. Where MDR is compressed, and interchange is zero or near zero, acquirers and payment-rail operators may still benefit from transaction volume, but credit issuers face weaker unit economics unless a separate credit monetization layer exists, such as higher consumer interest, merchant-funded BNPL fees, on-us ecosystems, or issuing credit cards.</p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>Two-sided market does not mean it involves two parties: a buyer and a seller. It means the total volume depends on the number of participants on the other side.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-2" href="#footnote-anchor-2" class="footnote-number" contenteditable="false" target="_self">2</a><div class="footnote-content"><p>Enabling merchants to accept cards.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-3" href="#footnote-anchor-3" class="footnote-number" contenteditable="false" target="_self">3</a><div class="footnote-content"><p>Availing buyer the means to send payments. Be it debit, credit, cheque, or cash.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-4" href="#footnote-anchor-4" class="footnote-number" contenteditable="false" target="_self">4</a><div class="footnote-content"><p>Payment methods involve credit facilities, such as credit cards.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-5" href="#footnote-anchor-5" class="footnote-number" contenteditable="false" target="_self">5</a><div class="footnote-content"><p>Individuals tend to be more price-sensitive than merchants.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-6" href="#footnote-anchor-6" class="footnote-number" contenteditable="false" target="_self">6</a><div class="footnote-content"><p>VISA or Mastercard charged acquirers a card scheme fee for using the network and an interchange fee to balance the costs.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-7" href="#footnote-anchor-7" class="footnote-number" contenteditable="false" target="_self">7</a><div class="footnote-content"><p>Merchant Discount Rate: the fee charged to merchants for processing the payment.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-8" href="#footnote-anchor-8" class="footnote-number" contenteditable="false" target="_self">8</a><div class="footnote-content"><p>National Bancard Corp. (NaBanco) v. Visa USA, Inc., 779 F. 2d 592 - Court of Appeals, 11th Circuit 1986.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-9" href="#footnote-anchor-9" class="footnote-number" contenteditable="false" target="_self">9</a><div class="footnote-content"><p>Issuer reimbursement fee: another name for interchange fee.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-10" href="#footnote-anchor-10" class="footnote-number" contenteditable="false" target="_self">10</a><div class="footnote-content"><p>The merchants are not acquired by the issuers.</p></div></div>]]></content:encoded></item><item><title><![CDATA[What 50 years of Singapore property prices tell us about risk]]></title><description><![CDATA[Why "average return" lies, why one quarter in 1993 still distorts a 35-year statistic, and what this means for anyone planning to buy property]]></description><link>https://www.zhizhi-gewu.com/p/what-50-years-of-singapore-property</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/what-50-years-of-singapore-property</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 25 Apr 2026 08:07:08 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!EQE6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>[Ideas are mine but analysis and texts are generated by AI]</p><p>Imagine two friends, Ahmad and Bao. Both decide to buy a private apartment in Singapore. Same building, same size. Ahmad buys in 1996 Q3. Bao buys in 2004 Q1. They both hold for five years.</p><p>Ahmad&#8217;s apartment, by 2001, is worth about 30% less than what he paid. He still has his mortgage. He&#8217;s underwater for almost a decade.</p><p>Bao&#8217;s apartment, by 2009, is worth about 45% more than what he paid. He&#8217;s the smart investor at every dinner party.</p><p>Same asset class. Same country. Same five-year holding period. Wildly different outcomes.</p><p>Most people, looking at this, will say &#8220;Ahmad got unlucky.&#8221; Or &#8220;Bao timed the market well.&#8221; Both are wrong, in an interesting way. The honest explanation is statistical, and once you see it, you&#8217;ll never read a &#8220;long-run average return&#8221; line in a property advert the same way again.</p><p>This piece walks through what 50 years of Singapore property data actually look like, what statisticians call fat tails, and the single most important &#8212; and most under-appreciated &#8212; fact about long-term property risk.</p><h2>The &#8220;average return&#8221; story</h2><p>Open any property-marketing brochure and you&#8217;ll see something like: &#8220;Singapore private property has appreciated by an average of 6.4% per year over the past 50 years.&#8221;</p><p>That&#8217;s not wrong. We pulled 50 years of quarterly data from SingStat &#8212; the URA Private Residential Property Price Index, from 1975 Q1 to 2025 Q4 &#8212; and the average comes out to about +1.57% per quarter, which annualises to roughly +6.4%. So far, so consistent.</p><p>The trouble is that the average is one of the least useful numbers you can compute about property returns. Here&#8217;s the actual time series:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!s3Wq!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!s3Wq!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png 424w, https://substackcdn.com/image/fetch/$s_!s3Wq!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png 848w, https://substackcdn.com/image/fetch/$s_!s3Wq!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png 1272w, https://substackcdn.com/image/fetch/$s_!s3Wq!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!s3Wq!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png" width="1300" height="650" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:650,&quot;width&quot;:1300,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:88384,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.zhizhi-gewu.com/i/195420447?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!s3Wq!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png 424w, https://substackcdn.com/image/fetch/$s_!s3Wq!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png 848w, https://substackcdn.com/image/fetch/$s_!s3Wq!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png 1272w, https://substackcdn.com/image/fetch/$s_!s3Wq!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81f7bc3-c102-45e3-bc15-3b5899e9e34f_1300x650.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The top panel is the price index. The bottom panel is the quarterly log-returns &#8212; basically, how much the index moved each quarter. Notice anything?</p><p>That bottom chart is not a tame, well-behaved process. It has periods of intense activity (the 1980&#8211;81 boom, the 1993&#8211;96 boom, the 1997&#8211;98 Asian crisis, the 2008 GFC dip and bounce) and long stretches of relative calm. The quarterly returns range from &#8722;15.2% (worst, 2009 Q1) to +24.4% (best, 1981 Q1). A single quarter can easily move the price by more than the average annual return.</p><p>Calling the average &#8220;+6.4%/yr&#8221; technically true and practically misleading is what statisticians call a Mediocristan-vs-Extremistan problem. We&#8217;ll come back to that.</p><h2>The thing we don&#8217;t talk about: kurtosis</h2><p>Here is the single most useful number for thinking about risk in fat-tailed worlds, and almost nobody outside finance ever encounters it: kurtosis.</p><p>(Pronounced &#8220;kur-TOH-sis.&#8221; Greek.)</p><p>It&#8217;s a measure of how much of the action is in the extremes versus the middle of a distribution. The bigger it is, the more the rare events dominate.</p><p>For a &#8220;normal&#8221; bell-curve distribution &#8212; the kind you learned about in school &#8212; kurtosis is 3.</p><p>For Singapore private residential property quarterly returns? 6.5.</p><p>For Singapore HDB resale flat returns? 18.2.</p><p>The HDB number is genuinely shocking. It is a direct symptom of one specific quarter: 1993 Q2, when HDB resale prices jumped by 27% in three months as a speculative boom rolled through public housing. That one observation, three decades later, still accounts for 74% of the total kurtosis of the entire HDB return distribution since 1990.</p><p>Translation in plain English: when you compute &#8220;the kurtosis of HDB returns&#8221; using 35 years of quarterly data, three-quarters of the answer comes from one quarter. The other 142 quarters together explain only the remaining 26% of it.</p><p>This is what &#8220;fat-tailed&#8221; means in practice. A single, unusual event dominates a statistic that&#8217;s supposed to summarise 35 years.</p><h2>A picture is worth a thousand standard deviations</h2><p>Here is the same point in chart form. The bars compare four Singapore property indices on four different &#8220;fat-tailedness&#8221; measures:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!THBf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!THBf!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png 424w, https://substackcdn.com/image/fetch/$s_!THBf!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png 848w, https://substackcdn.com/image/fetch/$s_!THBf!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png 1272w, https://substackcdn.com/image/fetch/$s_!THBf!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!THBf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png" width="1456" height="416" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:416,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:80306,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.zhizhi-gewu.com/i/195420447?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!THBf!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png 424w, https://substackcdn.com/image/fetch/$s_!THBf!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png 848w, https://substackcdn.com/image/fetch/$s_!THBf!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png 1272w, https://substackcdn.com/image/fetch/$s_!THBf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4ac5a381-49d2-4491-93aa-2095fcb60a6b_1820x520.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The red dashed lines show what a &#8220;normal-ish&#8221; distribution would look like.</p><p>Pearson kurtosis should be 3 for normal. SG indices range 5.6 to 18.2. Max quartic share &#8212; how much of the kurtosis is driven by one observation &#8212; should be about 0.05 for normal. SG indices range 0.31 to 0.74. Student-T fitted df, a sophisticated fat-tail measure where lower means fatter, with values near 2 indicating the underlying variance estimate is unreliable: SG indices 2.13 to 2.51. Hill &#945; tail exponent, where lower means heavier tail and below 4 means kurtosis itself is mathematically infinite in the population: SG indices 2.17 to 2.91.</p><p>Across every measure, every SG property index is in the &#8220;fat-tailed&#8221; zone. None of them resemble the bell curve that most financial planning quietly assumes.</p><h2>Why this matters, in dollars</h2><p>Most retirement planning, mortgage stress-testing, and &#8220;how much can I afford?&#8221; calculations implicitly assume returns behave like a normal distribution. They use words like &#8220;expected return &#177; standard deviation&#8221; and reason about probabilities the way you&#8217;d reason about coin flips.</p><p>In a normal-distribution world, a quarter that&#8217;s 5&#963; off the mean is essentially impossible. About 1 in 1.7 million.</p><p>In a fat-tailed world, what financial textbooks call a &#8220;5&#963; event&#8221; can be a 1-in-100 event, or even more frequent. The 1993 Q2 HDB jump is a 7&#963; event in a normal model. The 1981 Q1 URA spike is a ~5&#963; event. The 2009 Q1 GFC dip is a ~3.4&#963; event. We&#8217;ve had multiple of these in 50 years, in just one country, in one asset class.</p><p>What that actually means if you&#8217;re buying a Singapore property today: your downside in any single quarter could be 15% or more, even with no leverage. Your upside in any single quarter could be 25% or more. The &#8220;1-in-50-years&#8221; event is more like 1-in-10. Standard deviation, as a measure of &#8220;how much can this move,&#8221; is simply not informative.</p><p>But there&#8217;s a more important fact than any of these &#8212; and it&#8217;s the one I think almost nobody knows about.</p><h2>The real lesson: it&#8217;s clustering, not noise</h2><p>Here is the most underappreciated finding in all of this. The chart below is the headline diagnostic from Nassim Taleb&#8217;s Statistical Consequences of Fat Tails. We applied it to Singapore data:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!EQE6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!EQE6!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png 424w, https://substackcdn.com/image/fetch/$s_!EQE6!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png 848w, https://substackcdn.com/image/fetch/$s_!EQE6!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png 1272w, https://substackcdn.com/image/fetch/$s_!EQE6!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!EQE6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png" width="1456" height="1008" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/bc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1008,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:279826,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.zhizhi-gewu.com/i/195420447?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!EQE6!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png 424w, https://substackcdn.com/image/fetch/$s_!EQE6!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png 848w, https://substackcdn.com/image/fetch/$s_!EQE6!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png 1272w, https://substackcdn.com/image/fetch/$s_!EQE6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbc19487a-d729-45b4-bfd7-539792f66e2b_1690x1170.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Each panel is one Singapore property index. The blue line shows the kurtosis (fat-tailedness) of returns aggregated over different time windows &#8212; 1 quarter, 2 quarters, all the way to 12 quarters (3 years). The green dashed line shows the same calculation, but with the returns randomly reshuffled in time.</p><p>Reshuffling preserves the distribution of returns. Same average, same variance, same fat marginal tails. What it destroys is the order: the relationship between consecutive quarters.</p><p>Look at what happens. The blue line (real, actual time-ordered data) stays elevated. Even at 12-quarter aggregation, kurtosis is well above the &#8220;normal&#8221; baseline of 3. The green dashed line (reshuffled) drops to roughly 3 &#8212; Gaussian &#8212; by lag 8 or 10.</p><p>If property returns were truly fat-tailed but independent quarter-to-quarter, the blue and green lines would look the same. They don&#8217;t.</p><p>What this means: the multi-year fat-tailed risk in Singapore property is not primarily about how fat-tailed the quarterly distribution is. It&#8217;s about the fact that bad quarters cluster together, and good quarters cluster together. The market spends years at a time in a &#8220;boom regime&#8221; or a &#8220;bust regime.&#8221;</p><p>Visible in the data: </p><ul><li><p>the 1980&#8211;81 boom was 6 consecutive quarters of double-digit gains. </p></li><li><p>The 1993 HDB boom was two consecutive quarters of +27% then +18%. </p></li><li><p>The 1996&#8211;2004 URA bust was 10-quarter, ~45% peak-to-trough drawdown, after which the index stayed below the 1996 peak for over a decade. </p></li><li><p>The 2008 GFC dip was 4 consecutive negative quarters before recovery.</p></li></ul><p>This is the difference between two mental models. Model A says fat-tailed but independent &#8212; each quarter is an independent draw from a fat-tailed distribution; long-run drawdowns wash out via central-limit theorem; standard fat-tail stats describe the risk well; 5-year worst case is moderate. Model B says clustered booms and busts &#8212; quarters within a regime are correlated; regimes are persistent; long-run drawdowns compound because all the bad quarters land in the same window; you also need to model regime persistence; 5-year worst case is severe (45%+ drawdowns observed).</p><p>The data say Model B is correct.</p><h2>What this means for an actual buyer</h2><p>If you buy property today expecting &#8220;the long-run average return is 6.4% per year, so over 5 years I should average 32% appreciation,&#8221; you&#8217;re using Model A. Model A says 5 years of bad luck is a freak event.</p><p>Model B says: the 5-year outcome is dominated by what regime you&#8217;re in, which is largely determined by when you bought. It&#8217;s not about luck. It&#8217;s about timing relative to the regime.</p><p>Practical consequences:</p><ol><li><p><strong>Late-regime entries are more dangerous than they look.</strong> Buying at the top of a sustained price run (high transaction volume, optimistic narratives, rising leverage in the system) raises the conditional probability that you&#8217;re at the start of a bust regime. Buying at the bottom of a sustained drawdown (low transaction volume, pessimistic narratives, distressed listings) does the opposite. Same property, same physical asset &#8212; wildly different conditional 5-year outcomes.</p></li><li><p><strong>Holding period interacts with regime, not with calendar time.</strong> A 5-year holder who happens to enter the start of a boom regime exits at peak. A 5-year holder who enters at the start of a bust compounds losses. Same hold length, completely different experience.</p></li><li><p><strong>Leverage that&#8217;s &#8220;safe in normal times&#8221; can ruin you in a long bust.</strong> A 70% LTV ratio is a manageable risk in a stable regime &#8212; even with the occasional bad quarter. In a 7-year bust regime where prices fall 40%, the same 70% LTV means negative equity for years. You can&#8217;t refinance, you can&#8217;t sell at non-distressed prices, and you have to keep servicing the loan throughout. Th==e risk is not the worst single quarter; it&#8217;s the duration of the bust regime exceeding the buffer your finances can absorb.</p></li><li><p><strong>Stress-test using historical events, not multiples of &#963;.</strong> When your bank or your spouse asks &#8220;what&#8217;s the worst case?&#8221;, don&#8217;t say &#8220;two standard deviations down.&#8221; Say &#8220;what if 1996&#8211;2004 happens again?&#8221; That&#8217;s a 45% drawdown over 7 years with the property essentially impossible to sell at non-distressed prices throughout. If your finances can survive that, you can buy. If they can&#8217;t, you can&#8217;t, regardless of what the average suggests.</p></li><li><p><strong>Don&#8217;t outsource your regime-judgment to property gurus.</strong> Most property advice in Singapore is sold by people whose income depends on you transacting. They are structurally biased to believe whatever bullish or bearish narrative drives volume. Your actual cycle-positioning decision should rest on regime indicators (transaction volumes, price-to-rent ratios, mortgage-to-income ratios, supply pipelines, policy direction) &#8212; not on someone else&#8217;s confidence.</p></li></ol><h2>The bigger lesson</h2><p>You can extend this beyond property. Almost every domain you care about is fat-tailed in this same way. </p><ul><li><p>Equity markets have clustered crashes (1929, 1987, 2000, 2008, 2020)</p></li><li><p>Career outcomes are dominated by a few breakthrough years. </p></li><li><p>Most of your lifetime medical risk is concentrated in a few specific events.</p></li><li><p>A small fraction of relationships carry most of the long-term consequence.</p></li></ul><p>The intuitive ways we think about averages and volatility &#8212; trained by exam questions about coin flips and dice &#8212; break down in these domains. The standard deviation underestimates the spread, the mean is dominated by outliers, and &#8220;averaging out over the long run&#8221; doesn&#8217;t happen because the long run is dominated by clustered regimes, not by independent draws.</p><h2>Want to verify this yourself?</h2><p>Here&#8217;s the entire analysis pipeline. Less than 100 lines of Python. The data is publicly available &#8212; no special access required.</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:null}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">
import json
import urllib.request
import numpy as np
import pandas as pd
from scipy import stats

# 1. Fetch URA Private Residential PPI from SingStat (public API)
URL = "https://tablebuilder.singstat.gov.sg/api/table/tabledata/M212261?limit=5000"
data = json.loads(urllib.request.urlopen(URL).read())
row = next(r for r in data["Data"]["row"] if r["rowText"] == "Residential Properties")
ppi = pd.Series(
    [float(c["value"]) for c in row["columns"]],
    index=pd.PeriodIndex(
        [pd.Period(f"{c['key'].split()[0]}Q{c['key'].split()[1][0]}", freq="Q")
         for c in row["columns"]],
        freq="Q",
    ),
)

# 2. Compute quarterly log-returns
r = np.log(ppi / ppi.shift(1)).dropna().values

# 3. Headline statistics
print(f"n = {len(r)} quarterly observations, 1975Q1 to 2025Q4")
print(f"mean per quarter   = {r.mean():+.4f}")
print(f"Pearson kurtosis   = {stats.kurtosis(r, fisher=False):.2f}  (Gaussian = 3)")

# 4. Single-event dominance test
x4 = r ** 4
print(f"Max single-quarter share of total x^4: {x4.max() / x4.sum():.3f}")

# 5. Clustering test (Taleb's Fig 10.2)
def kurt_at_lag(returns, lag):
    n = len(returns) // lag
    aggregated = returns[:n*lag].reshape(n, lag).sum(axis=1)
    return stats.kurtosis(aggregated, fisher=False)

print(f"raw,        lag=12: {kurt_at_lag(r, 12):.2f}")
rng = np.random.default_rng(42)
shuffled = []
for _ in range(200):
    s = r.copy(); rng.shuffle(s)
    shuffled.append(kurt_at_lag(s, 12))
print(f"reshuffled, lag=12: {np.mean(shuffled):.2f}  (mean of 200 shuffles)")
```

When you run it, you'll see the raw lag-12 kurtosis stays well above 3, but the reshuffled version drops to about 2.7 &#8212; exactly the volatility-clustering signature this article describes.

Disclosure

Both this article and the Python analysis code were generated by Claude, Anthropic's AI assistant. The data is from SingStat (Singapore's Department of Statistics) and is publicly available. The methodology follows Chapter 10 of Nassim Nicholas Taleb's Statistical Consequences of Fat Tails (2020), particularly Figure 10.2 on the role of volatility clustering in apparent fat-tailedness.

I ran every test described, generated every chart from the actual data, and have full reproducibility. If anything looks wrong, please tell me &#8212; fat-tail analysis is famously easy to get subtly wrong, and a second pair of eyes is welcome.</code></pre></div>]]></content:encoded></item><item><title><![CDATA[The Hidden Interest Rate in Your Insurance Bill]]></title><description><![CDATA[A mental shortcut for comparing payment plans.]]></description><link>https://www.zhizhi-gewu.com/p/the-hidden-interest-rate-in-your</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/the-hidden-interest-rate-in-your</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 19 Apr 2026 02:15:45 GMT</pubDate><content:encoded><![CDATA[<p>You&#8217;re 64 years old, flipping through AIA&#8217;s health insurance brochure. The same basic plan shows four prices for the same coverage:</p><ul><li><p><strong>Annual:</strong> HKD 72,440</p></li><li><p><strong>Semi-annual:</strong> HKD 36,944 &#215; 2 = 73,888</p></li><li><p><strong>Quarterly:</strong> HKD 20,280 &#215; 4 = <strong>81,120</strong></p></li><li><p><strong>Monthly:</strong> HKD 6,400 &#215; 12 = 76,800</p></li></ul><p>Two things jump out. First, monthly is only about 6% more than annual &#8212; tempting for the cash-flow relief. Second, <em>quarterly is the most expensive of the four</em> &#8212; more than monthly. Strange. Is it just a weird pricing quirk, or is the insurer telling you something?</p><p>The 6% on monthly isn&#8217;t a fee &#8212; it&#8217;s <strong>interest</strong>, and you&#8217;re paying it on money still sitting in your pocket. So the right question isn&#8217;t &#8220;how much extra?&#8221; but &#8220;<strong>what interest rate am I paying to delay?</strong>&#8220;</p><h2>The exact answer is IRR. The useful one lives in your head.</h2><p>Let <em>m</em> be the monthly payment and <em>r</em> the monthly rate. For the installment stream to be worth the same as paying <em>P</em> upfront today:</p><blockquote><p><em>P = m + m/(1+r) + m/(1+r)&#178; + &#8943; + m/(1+r)&#185;&#185;</em></p></blockquote><p>Solving this exactly is what IRR does. Instead, use the approximation anyone with a calculus class already knows:</p><blockquote><p><em>1 / (1+r)&#8319; &#8776; 1 &#8722; nr</em> (small <em>r</em>)</p></blockquote><p>Plug it in and the messy geometric series collapses into a clean arithmetic one:</p><blockquote><p><em>P &#8776; m &#183; [ 1 + (1 &#8722; r) + (1 &#8722; 2r) + &#8943; + (1 &#8722; 11r) ] = 12m &#8722; m&#183;r&#183;(1 + 2 + &#8943; + 11)</em></p></blockquote><p>That last sum is <strong>66</strong>. So:</p><blockquote><p><em>P &#8776; 12m &#8722; 66 m r = 12m &#183; (1 &#8722; 5.5 r)</em></p></blockquote><p>Here&#8217;s where the intuition crystallizes. That <strong>5.5</strong> isn&#8217;t arbitrary &#8212; it&#8217;s the <strong>average number of months you delay a payment</strong>. The first payment is on day zero, the last is at month 11; on average, each dollar is delayed 5.5 months.</p><p>Now flip it into years. Let <em>R = 12r</em> be the annual rate, and call <em>d&#772; = 5.5 / 12 &#8776; 0.458</em> the <strong>average delay in years</strong>. Then:</p><blockquote><p><em>P &#8776; A &#183; (1 &#8722; R &#183; d&#772;)</em>, where <em>A = 12m</em> is the total you pay.</p></blockquote><p>One rearrangement gives the rule:</p><blockquote><p><strong>R &#8776; markup / d&#772;</strong>, with <em>markup = (A &#8722; P) / A</em>.</p></blockquote><p><strong>The implied rate is the markup divided by the average delay in years.</strong> Nothing more.</p><h2>The cheat sheet</h2><p>For <em>N</em> equal payments spaced evenly over a year, the average delay is the arithmetic-sum shortcut applied once and converted to years:</p><blockquote><p><em>d&#772; = (0 + 1 + &#8943; + (N&#8722;1)) / N &#215; 1/N yr = (N &#8722; 1) / (2N) yr</em></p></blockquote><ul><li><p><strong>Monthly</strong> (N = 12): d&#772; = 11/24 &#8776; 0.458 yr &#8594; multiplier &#8776; <strong>2.2&#215;</strong> (markup &#247; 0.458)</p></li><li><p><strong>Quarterly</strong> (N = 4): d&#772; = 3/8 = 0.375 yr &#8594; multiplier &#8776; <strong>2.7&#215;</strong> (markup &#247; 0.375)</p></li><li><p><strong>Semi-annual</strong> (N = 2): d&#772; = 1/4 = 0.25 yr &#8594; multiplier = <strong>4&#215;</strong> (markup &#247; 0.25)</p></li></ul><p>Notice the pattern: fewer payments &#8594; shorter average delay &#8594; a given markup implies a <em>higher</em> rate. A 3% markup on a semi-annual plan is expensive in rate terms (~12%); the same 3% on a monthly plan is ~6.5%.</p><h2>Back to the AIA plan</h2><p>Apply the rule <em>R &#8776; markup / d&#772;</em>:</p><ul><li><p><strong>Semi-annual:</strong> markup 2.0% &#215; 4 = <strong>~8% per year</strong></p></li><li><p><strong>Monthly:</strong> markup 5.7% &#215; 2.2 = <strong>~13% per year</strong></p></li><li><p><strong>Quarterly:</strong> markup 10.7% &#215; 2.7 = <strong>~29% per year</strong></p></li></ul><p>Suddenly the strange quarterly pricing makes sense. The insurer isn&#8217;t making a mistake &#8212; they&#8217;re quietly charging you roughly <strong>30% per year</strong> to stretch payments over nine months. The monthly plan is a far better deal (and annual is better still if you have the cash).</p><p>Now you can actually choose. If your alternative is a credit card at 24%, the monthly plan&#8217;s ~13% is cheaper money &#8212; take it. If your cash is sitting in a HK deposit at 4%, pay annual and pocket the 9-point spread. And whatever you do, <strong>don&#8217;t pick quarterly</strong> unless you have no other option, because you&#8217;re borrowing at subprime rates for the privilege.</p><h2>When it breaks</h2><p>The approximation is tight when <em>nr</em> is small. For semi-annual and monthly above, it matches the exact IRR within a percentage point &#8212; 7.8% vs 8.3%, 12.4% vs 13.8%. For quarterly, where the true rate is ~37%, the linearization starts to fail and our shortcut underestimates by about 8 points. The rule of thumb is: trust the shortcut up to ~20% annualized; beyond that, it still tells you the <em>direction</em> (&#8221;this is expensive&#8221;) but you&#8217;ll want a spreadsheet for the exact number.</p><p>It also breaks when the schedule isn&#8217;t evenly spaced &#8212; a big deposit plus small monthlies, or a ballooning final payment. Same fix: spreadsheet.</p><p>But for the ordinary case of comparing payment frequencies on an insurance quote: <strong>divide the markup by the average delay in years, and you have your interest rate.</strong> It takes five seconds and it&#8217;s usually the difference between a good deal and a quietly terrible one.</p>]]></content:encoded></item><item><title><![CDATA[When to Use a Blacklist, and When to Use a Rule]]></title><description><![CDATA[After joining the anti-fraud team, I noticed that blacklists are used far more extensively than they were in credit underwriting.]]></description><link>https://www.zhizhi-gewu.com/p/when-to-use-a-blacklist-and-when</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/when-to-use-a-blacklist-and-when</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 04 Apr 2026 04:16:20 GMT</pubDate><content:encoded><![CDATA[<p>After joining the anti-fraud team, I noticed that blacklists are used far more extensively than they were in credit underwriting. That observation led me to reflect on the difference between a blacklist and a rule.</p><p>For readers outside the risk management field, a blacklist is a finite set of objects, such as a user ID, device fingerprint, phone number, or identity card number. Entities on the blacklist are not allowed to use the product. A rule, by contrast, is a decision function: it takes an input and produces an outcome, such as pass, reject, or review.</p><p>Mathematically, both can be expressed as indicator functions:</p><p>R(x) = 1[C(x)]</p><p>where C(x) is some condition on x, and</p><p>R(x)=1[X in B]</p><p>where B is a list.</p><p>In this sense, a blacklist is simply a special case of a general rule in which the condition is set membership. Mathematically, both are the same type of object: they map an input to a decision. In practice, however, the distinction remains useful. The term <em>rule</em> usually refers to non-list-like logic, while <em>blacklist</em> refers to list-like logic based on explicit membership. The real question, then, is not whether a blacklist is a rule, but when we should rely on list-like rules and when non-list-like rules are more appropriate.</p><p>A blacklist is like a memory of bad actors. Once a bad actor is identified, the goal is to prevent them from exploiting the product again. But this only works under two conditions: first, we can identify them reliably; second, the identifier is not easy to replace or rotate. For that reason, blacklists are most suitable for high-confidence, confirmed cases tied to relatively durable identifiers. If our confidence is low, or if the identifier can be changed easily, a blacklist may do more harm than good by blocking legitimate users while doing little to stop the bad actor in the long run.</p><p>A non-list-like rule, by contrast, captures a more general pattern of risk. Someone rejected by the rule today may not be rejected tomorrow, because the decision depends on current attributes or behavior rather than fixed membership in a list. This makes rules more suitable when the signal is weaker, more probabilistic, or tied to identifiers that can be changed easily.</p><p>In practice, however, teams often blur the boundary between the two. Low-confidence signals or easily rotated identifiers are sometimes added to blacklists, which can create high false-positive rates. Conversely, even high-confidence bad actors are sometimes handled only through dynamic rules, leaving room for repeated breaches once the pattern changes or the rule is circumvented.</p><p>The key is to match the tool to the nature of the signal. A blacklist works best when the signal is strong and the identity is durable. A rule works better when the signal is less certain or when the adversary can easily change identifiers. Although the two are mathematically similar, they play different operational roles: a blacklist acts as memory, while a rule acts as generalized reasoning. Confusing the two can either block too many good users or allow known bad actors to return.</p>]]></content:encoded></item><item><title><![CDATA[If You Cannot Create It, You Don't Understand It — Even with AI"]]></title><description><![CDATA[I recently built a tiny autograd engine from scratch &#8212; just a hundred lines of Python that can compute gradients through a computation graph.]]></description><link>https://www.zhizhi-gewu.com/p/if-you-cannot-create-it-you-dont</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/if-you-cannot-create-it-you-dont</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 29 Mar 2026 04:00:14 GMT</pubDate><content:encoded><![CDATA[<p>I recently built a tiny autograd engine from scratch &#8212; just a hundred lines of Python that can compute gradients through a computation graph. The kind of thing an LLM could spit out in seconds. But I didn&#8217;t let it. I wrote every line myself, hit bugs, traced gradients by hand on paper, and asked the AI to check my work &#8212; not to do it for me. And honestly? It was the most satisfying learning experience I&#8217;ve had in a while.</p><h2>Reflections on Learning with AI</h2><p><strong>LLMs can hinder learning if you let them: </strong>Learning is fundamentally hard because it requires working your brain. An LLM can provide solutions instantly, but that bypasses the struggle where real understanding is built. Think of the LLM as a teacher sitting beside you &#8212; the teacher cannot complete the task for the student. You still have to do the thinking.</p><p><strong>LLMs can act as a teacher, but only if you drive the conversation:</strong> They can verify your understanding, answer questions, and generate practice problems. However, current LLMs are not proactive teachers &#8212; they respond, they don&#8217;t lead. It takes skill to prompt the LLM effectively, which is challenging, especially for younger students who may not yet be comfortable with self-directed learning.</p><p><strong>LLMs are better at details than intuition:</strong> They tend to explain step-by-step solutions well but struggle to convey the big picture and the intuition behind the concepts. For example, Section 1 (The Big Picture) of these notes was my own mental model, polished by the LLM &#8212; not generated by it.</p><p><strong>Writing code by hand is irreplaceable:</strong> Having the opportunity to implement something yourself is deeply satisfying and solidifies understanding in a way that reading or prompting never can. As the saying goes, <strong>&#8220;What I cannot create, I do not understand.&#8221;</strong> (Richard Feynman). The bugs you hit, the edge cases you miss, the moments where it finally clicks &#8212; that&#8217;s where the real learning happens.</p><p>Attached the study note on autogradiant decent prepared together with LLM:</p><div class="file-embed-wrapper" data-component-name="FileToDOM"><div class="file-embed-container-reader"><div class="file-embed-container-top"><image class="file-embed-thumbnail-default" src="https://substackcdn.com/image/fetch/$s_!0Cy0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack.com%2Fimg%2Fattachment_icon.svg"></image><div class="file-embed-details"><div class="file-embed-details-h1">Study Notes</div><div class="file-embed-details-h2">137KB &#8729; PDF file</div></div><a class="file-embed-button wide" href="https://www.zhizhi-gewu.com/api/v1/file/ad05faf1-1534-467d-bf0e-a0f3c01cfabe.pdf"><span class="file-embed-button-text">Download</span></a></div><a class="file-embed-button narrow" href="https://www.zhizhi-gewu.com/api/v1/file/ad05faf1-1534-467d-bf0e-a0f3c01cfabe.pdf"><span class="file-embed-button-text">Download</span></a></div></div><p></p><p></p>]]></content:encoded></item><item><title><![CDATA[Can AI Agents Win a Modeling Challenge? A Replicable Experiment]]></title><description><![CDATA[To get the most out of AI agents, we need to remove human bottlenecks and increase leverage.]]></description><link>https://www.zhizhi-gewu.com/p/can-ai-agents-win-a-modeling-challenge</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/can-ai-agents-win-a-modeling-challenge</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Wed, 25 Mar 2026 15:21:54 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!R4ip!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="pullquote"><p>To get the most out of AI agents, we need to remove human bottlenecks and increase leverage.</p></div><p><strong>If you want to jump straight to the implementation, the full code is here:</strong> <a href="https://github.com/kychanbp/monee_auto_ml_comp_2026_demo">Github</a>.</p><h2>What Was Really Being Tested</h2><p>My company recently held a modeling competition in which participants were allowed to use only AI tools, without writing any code themselves. The objective was straightforward: maximize the AUC of a supervised learning task.</p><p>In tabular supervised learning, the practical toolkit is already quite mature. Gradient boosting models remain the dominant high-performance baseline, and in real-world applications the largest gains often come not from changing the model class, but from accumulating more data, engineering better features, and defining targets that better reflect business objectives. Model fine-tuning can help at the margin, but it rarely produces step-change improvements. Even in sequential modeling, much of the value can be understood as learning richer representations of the underlying behavior and structure.</p><p>For that reason, the most interesting question was not whether an LLM could write code to implement ideas that human practitioners had already provided. If humans specify the feature candidates, the modeling tricks, and the overall direction, then the exercise becomes largely a test of coding and execution. That is useful, but not especially interesting. A more meaningful test is whether an LLM can autonomously discover promising ideas for itself: what features to construct, which techniques are worth trying, how to interpret intermediate results, and how to iterate toward better performance under a clear objective.</p><p>Seen from this perspective, the purpose of the competition, in my own opinion, was not to discover a fundamentally new classification algorithm nor implementing solutions based on human insights. It was to test whether an LLM could independently reproduce the practical workflow behind strong credit risk modeling: generating hypotheses, engineering useful features, running experiments, learning from feedback, and improving performance through iteration. Put differently, if the playbook used by experienced practitioners is only partially specified, how much of it can an LLM recover on its own?</p><p>This also made the exercise a test of instruction design. Beyond model performance, it offered a way to understand how tasks should be framed so that an LLM can explore the solution space productively rather than simply execute a predefined recipe.</p><h2>The Experiment Setup</h2><p>At first glance, the objective may seem clear enough: give the LLM a target metric and ask it to iterate on its own. But things work more smoothly in human teams because people already share a large amount of tacit context: what counts as a valid experiment, what shortcuts are unacceptable, how performance should be evaluated, and when a result is worth keeping. For an LLM, many of these assumptions have to be made explicit.</p><p>To make the exercise meaningful, we needed to specify a set of operating rules that human teams would often leave implicit.</p><p>The experimental setup was not fully specified from the start; as I observed the agent&#8217;s behavior, I iteratively refined and steered it. For example, in the later stages, I found that the acceptance threshold had become too strict.</p><p><strong>Scope.</strong> We had to define what the agent was allowed to modify, what it was not allowed to touch, and what kinds of actions were permitted or prohibited.</p><p><strong>Integrity.</strong> The agent could use only data available before the application date. It also had to check for leakage before proceeding, especially when a single feature appeared to perform suspiciously well.</p><p><strong>Evaluation.</strong> The evaluation methodology had to be fixed in advance. Otherwise, the LLM could improve reported scores simply by changing the validation setup rather than improving the model itself.</p><p><strong>Promotion criteria.</strong> We needed to define what magnitude of improvement was sufficient for a new approach to be accepted and carried forward.</p><p><strong>Logging.</strong> Like humans, LLMs do not naturally produce good documentation unless asked to do so. To make their work inspectable&#8212;and to give the agent a usable record of its own progress&#8212;we had to explicitly require logging.</p><p><strong>Resource constraints.</strong> The agent needed rules for efficient experimentation: cache generated features for reuse, avoid recomputing unnecessarily, explore ideas in parallel where possible, and operate within limits on the number and scale of experiments.</p><p><strong>Simplicity.</strong> Some human judgment still had to be encoded into the setup. In general, we preferred simpler models when they delivered performance comparable to more complex alternatives.</p><p><strong>Stopping criteria.</strong> We also had to define when the agent should stop iterating, rather than continuing to search indefinitely for marginal gains.</p><p>These rules were not just administrative details. They were part of the experiment itself. If the goal was to test whether an LLM could behave like a disciplined modeler, then the environment had to specify the constraints under which disciplined modeling takes place.</p><p>You can find the full instruction to the LLM <a href="https://github.com/kychanbp/monee_auto_ml_comp_2026_demo/blob/main/program.md">here</a>.</p><h2>How the LLM Iterated</h2><p>The LLM spent around 48 hours to reach the final result. In the first 24 hours, I did not prompt the LLM anything and it reached 0.793424 AUC at round 31. The, the LLM stucked for consecutive 12 rounds. I prompted the LLM to leverage websearch to search for ideas, although I already stated that in the original prompt and it preferred relying on its internal knowledge.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!R4ip!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!R4ip!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png 424w, https://substackcdn.com/image/fetch/$s_!R4ip!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png 848w, https://substackcdn.com/image/fetch/$s_!R4ip!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png 1272w, https://substackcdn.com/image/fetch/$s_!R4ip!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!R4ip!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png" width="1456" height="607" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/cfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:607,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Experiment Progress&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Experiment Progress" title="Experiment Progress" srcset="https://substackcdn.com/image/fetch/$s_!R4ip!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png 424w, https://substackcdn.com/image/fetch/$s_!R4ip!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png 848w, https://substackcdn.com/image/fetch/$s_!R4ip!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png 1272w, https://substackcdn.com/image/fetch/$s_!R4ip!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfa22e0a-19b9-4404-ab2a-68a0310973b0_1800x750.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h2>Thoughts</h2><p>With clearly defined objectives and boundaries, an LLM can iterate on its own and uncover most of the early low-hanging fruit much faster than a human can. However, more distant ideas&#8212;such as KNN-based approaches or sequential modeling, which require a more creative leap&#8212;seem harder for it to discover. That said, the LLM only spent two days exploring the solution space. Without internet access, it is not obvious that a human could have found those ideas within the same time frame either. My guess is that the top-performing team spent far more than two days to achieve its breakthrough, and likely involved multiple team members.</p><p>What stands out most is how much cheaper it becomes to try new ideas. The cost of experimentation drops sharply in terms of both time and human effort. With its coding ability, the LLM can translate modeling ideas into working code extremely quickly. Generating roughly 300 additional features, running hyperparameter tuning, stacking multiple models, and building sequential representations of the data&#8212;all within two days&#8212;would be expensive for a human team. To do the same amount of work, we might need five to ten experienced practitioners. In my case, I did not even fully use the token budget of Claude&#8217;s USD 100 max plan.</p><p>Will humans lose their jobs? Partly, yes. LLMs will increasingly replace a large share of the coding, pipeline construction, and some feature engineering work. But they still need humans to provide direction. The human role will shift toward defining the scope, constraints, and objectives of the task, and then validating the outputs. At least for now, humans also seem to have better taste&#8212;in the sense of searching the solution space more efficiently and recognizing which directions are truly promising. We sometimes hear stories of LLMs surfacing obscure old papers that solve a problem outright, which suggests that this advantage may not always hold. But on average, I still think humans with domain knowledge retain an edge.</p><p>Will that remain true as we use LLMs more and progressively transfer domain knowledge into them? I do not know. If forced to give a rough estimate, I would guess that within five to ten years, LLMs may surpass humans even in the discovery of previously unknown solutions.</p><p>What seems much clearer is that, over the next five years, human-plus-AI will dominate human-alone workflows. The productivity gain is easily an order of magnitude for many existing tasks, and in some cases effectively unbounded because AI enables work that would not have been attempted otherwise. If you are not using AI every day to learn new things, you are likely falling behind. If you are not using it to accelerate coding-related work, you are almost certainly moving much more slowly. And if you are not using it to help generate new ideas and solutions, you may already be at a meaningful disadvantage. Once these effects compound, the gap between human-only and human-plus-AI workflows becomes very large.</p><h2>Next Step</h2><p>If the current limitation is the LLM&#8217;s &#8220;taste,&#8221; then the next frontier is not just better models, but better ways of searching the solution space. The key challenge is to help the LLM explore more intelligently, so that it can identify promising directions earlier instead of relying mainly on broad trial and error. Early thoughts including the use of tree-of-thoughts and Monte Carlo Tree Search (MCTS) which I will try to implement in next week.</p><p>I also need to better maximize the available token budget. During this project, the agent never came close to exhausting the token allowance of the Claude Max ($100/month) plan. In practice, a considerable amount of time was spent waiting for model runs to finish while the agent was otherwise idle, suggesting that I did not make full use of the available capacity.</p><h2>Recommend Readings</h2><iframe class="spotify-wrap podcast" data-attrs="{&quot;image&quot;:&quot;https://i.scdn.co/image/ab6765630000ba8a5f26cc6401340f282d22e1dd&quot;,&quot;title&quot;:&quot;Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI&quot;,&quot;subtitle&quot;:&quot;Conviction&quot;,&quot;description&quot;:&quot;Episode&quot;,&quot;url&quot;:&quot;https://open.spotify.com/episode/3u1XHEsAuK1VXRr81RzQDV&quot;,&quot;belowTheFold&quot;:true,&quot;noScroll&quot;:false}" src="https://open.spotify.com/embed/episode/3u1XHEsAuK1VXRr81RzQDV" frameborder="0" gesture="media" allowfullscreen="true" allow="encrypted-media" loading="lazy" data-component-name="Spotify2ToDOM"></iframe>]]></content:encoded></item><item><title><![CDATA[How Singapore Savings Bonds Work: A Financial Engineering Perspective]]></title><description><![CDATA[Not investment advice]]></description><link>https://www.zhizhi-gewu.com/p/how-singapore-savings-bonds-work</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/how-singapore-savings-bonds-work</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 15 Mar 2026 09:07:39 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!BuIF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Crossing the 30s mark, financial life has changed significantly, mainly due to increased liabilities. While human capital remains the primary cash-generating asset, as was common in the early 30s, it is important to build greater robustness into the system because all income depends on continuous production by the brain and body. Robustness can be achieved through downside volatility mitigation and diversification. Insurance and cash reserves can help cap the downside, and financial capital diversifies human capital risk to some extent. This article discusses building cash reserves and, in particular, the characteristics of the Singapore Savings Bonds (SSB).</p><h2>Cash is &#8220;Still&#8221; the King</h2><p>Many financial advisors say cash is trash because the government can &#8220;print&#8221; money and that cash will depreciate relative to other assets. Therefore, cash is a poor asset to hold. However, it is not always the case. If equity goes down by 18%, the same amount of cash can now buy 22% more equity (1/(1-18%) - 1) &#8212; cash gains in <em>relative</em> purchasing power even though its nominal value stays the same. In particular, during monetary tightening, bond prices fall while cash yields rise, making cash an attractive asset to hold. More generally, cash earns interest when treated as an investable asset rather than a medium of exchange, and its yield roughly follows the short end of the yield curve.</p><h2>Singapore Savings Bonds</h2><p>SSB is a 10-year government bond issued monthly by MAS, available to individuals in multiples of $500. What makes it unique is its structure: SSB is a single instrument with a guaranteed principal that replicates the SGS yield curve at every holding period &#8212; prioritising the step-up feature when the curve shape doesn&#8217;t naturally allow it. To my knowledge, this structure offers a few advantages:</p><ul><li><p><strong>Flexibility in holding period:</strong> Investors do not have to commit to a fixed holding period, unlike with fixed deposits and bonds. SSBs can be redeemed at par in any given month with no penalty.</p></li><li><p><strong>No price risk:</strong> Unlike a bond, which trades on the secondary market and whose price can drop below par value, the SSB principal is guaranteed.</p></li><li><p><strong>No reinvestment risk:</strong> The coupons are guaranteed and locked in at issuance. For 6-month or 1-year T-bills, when they mature, investors must reinvest at the then-available rate, which may be lower.</p></li></ul><p>Effectively, SSB offers two embedded protections. The first is a free put option at par, allowing the investor to &#8220;sell&#8221; it back to the government at face value at any time &#8212; this protects when rates rise. The second is a free forward rate agreement on the full yield curve, locking in returns for up to 10 years &#8212; this protects when rates fall. The limitation, however, is the investment cap ($50,000 per issue and $200,000 per individual).</p><h2>How SSB Rates Reflect the Yield Curve</h2><p>To understand how SSB coupons are derived, it helps to look at the underlying SGS yield curve across different rate environments. The chart below shows three panels: the SGS benchmark yields (input), the resulting SSB coupon rates (output), and the average returns investors actually earn.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!BuIF!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!BuIF!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png 424w, https://substackcdn.com/image/fetch/$s_!BuIF!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png 848w, https://substackcdn.com/image/fetch/$s_!BuIF!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png 1272w, https://substackcdn.com/image/fetch/$s_!BuIF!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!BuIF!,w_2400,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png" width="1200" height="1502.4725274725274" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:false,&quot;imageSize&quot;:&quot;large&quot;,&quot;height&quot;:1823,&quot;width&quot;:1456,&quot;resizeWidth&quot;:1200,&quot;bytes&quot;:1411031,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.zhizhi-gewu.com/i/191002074?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:&quot;center&quot;,&quot;offset&quot;:false}" class="sizing-large" alt="" srcset="https://substackcdn.com/image/fetch/$s_!BuIF!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png 424w, https://substackcdn.com/image/fetch/$s_!BuIF!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png 848w, https://substackcdn.com/image/fetch/$s_!BuIF!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png 1272w, https://substackcdn.com/image/fetch/$s_!BuIF!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F732f5cc1-ec16-42f0-9f4c-f6ebb572223b_1782x2231.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><strong>Panel 1 &#8212; SGS Yield Curves (the input)</strong></p><p>The yield curve shape has changed dramatically across eras</p><ul><li><p>2015-2017 (purple): Steep upward slope &#8212; short rates ~1%, long rates ~2.2%</p></li><li><p>2020-2021 (teal): Lowest across all tenors &#8212; near-zero short rates from COVID easing</p></li><li><p>2022-2024 (red): Inverted &#8212; short rates (~3.5%) are higher than long rates (~3%) from aggressive rate hikes</p></li><li><p>2025-2026 (orange): Re-steepening as short rates fall faster than long rates</p></li></ul><p>Notably, all eras converge to a narrower range at 10Y (~1.4-3%) than at 1Y (~0.3-3.5%), reflecting the fact that long-term rate expectations are more stable than short-term policy rates.</p><p> <strong>Panel 2 &#8212; SSB Coupons (the output)</strong></p><ul><li><p>The step-up mechanism transforms the yield curve shape into a monotonically increasing schedule</p></li><li><p>Inverted curves (red) become flat coupons &#8212; the monotonicity adjustment compresses everything to the 10Y level</p></li><li><p>Steep curves (purple) produce dramatic step-ups &#8212; from ~1% to ~3.5%</p></li><li><p>The Y10 coupon is always higher than the Y10 yield because it compensates for the lower early coupons</p></li></ul><p> <strong>Panel 3 &#8212; Average Returns (what investors actually earn)</strong></p><ul><li><p>This is the smoothed version of the coupon schedule &#8212; always monotonically increasing and more gradual</p></li><li><p>At 10Y, it matches the SGS yield in Panel 1 &#8212; confirming the design works</p></li><li><p>At shorter tenors, it matches SGS yields only when no monotonicity adjustment is needed</p></li></ul><p> <strong>Key takeaway</strong></p><p>The three panels tell the story of a single financial engineering pipeline: a market yield curve (which can be any shape &#8212; steep, flat, inverted) gets transformed into an investor-friendly product that always steps up, always returns principal, and always delivers the 10Y market return. The price of this guarantee is visible in the red era &#8212; when the curve inverted, short-term SSB holders earned well below the prevailing T-bill rate because the step-up constraint sacrificed short-tenor returns to preserve the monotonicity feature.</p><h2>Methodology</h2><p>For those interested in the methodology for deriving the SSB coupon from SGS benchmark rates, you may refer to this <a href="https://www.mas.gov.sg/-/media/mas/sgs/sgs-announcements-pdf/ssb-pdf/ssb-technical-specifications/ssb-technical-specifications.pdf?utm_source=chatgpt.com">MAS paper.</a> Here is a high-level skeleton: </p><p><strong>Step 1 &#8212; Get Benchmark Yields</strong></p><ul><li><p>Download daily SGS benchmark yields from <a href="https://eservices.mas.gov.sg/statistics/fdanet/BenchmarkPricesAndYields.aspx">MAS Benchmark Prices and Yields</a></p></li><li><p>Specifically: 1-Year T-Bill Yield, 2-Year Bond Yield, 5-Year Bond Yield, 10-Year Bond Yield</p></li><li><p>Compute the simple average of all trading days from month M-2 (two months before the SSB issue month)</p></li></ul><p><strong>Step 2 &#8212; Interpolate the Full Yield Curve</strong></p><ul><li><p>Use a hermite spline to fill in the missing tenors (3Y, 4Y, 6Y, 7Y, 8Y, 9Y)</p></li><li><p>Now you have 10 par yields: Y1, Y2, ..., Y10</p></li></ul><p><strong>Step 3 &#8212; Bootstrap Discount Factors</strong></p><ul><li><p>Convert par yields into discount factors (the present value of $1 received at each future year)</p></li><li><p>DF1 = 1/(1+Y1), then each subsequent DF is solved from the no-arbitrage pricing equation</p></li></ul><p><strong>Step 4 &#8212; Solve for Step-Up Coupons</strong></p><ul><li><p>Find coupon rates C1, C2, ..., C10 such that the bond prices are at par for every holding period</p></li><li><p>This means: an investor who holds for N years and redeems at par earns the same return as an N-year SGS bond</p></li><li><p>C1 = Y1, then each subsequent coupon is solved forward using the discount factors</p></li></ul><p><strong>Step 5 &#8212; Enforce Monotonicity</strong></p><ul><li><p>If the raw coupons don&#8217;t step up (e.g. inverted yield curve), adjust them:</p><ul><li><p>Minimize the pricing error across all tenors</p></li><li><p>Subject to: coupons never decrease, 10Y return is preserved exactly</p></li></ul></li><li><p>Round to 2 decimal places</p></li></ul><p><strong>Data Sources</strong></p><ul><li><p><strong>SGS benchmark yields</strong>: <a href="https://eservices.mas.gov.sg/statistics/fdanet/BenchmarkPricesAndYields.aspx">MAS Benchmark Prices and Yields</a></p></li><li><p><strong>Published SSB rates</strong>: <a href="https://eservices.mas.gov.sg/statistics/fdanet/StepUpInterest.aspx">MAS Step-Up Interest Rates</a> (127 bonds, Oct 2015 &#8211; Apr 2026)</p></li><li><p><strong>Technical specification</strong>: <a href="https://www.mas.gov.sg/bonds-and-bills/Singapore-Savings-Bonds">SSB Technical Specifications</a>, Section 4</p></li></ul><p></p><p></p>]]></content:encoded></item><item><title><![CDATA[Fraud Management and the Pricing of Tail Risk]]></title><description><![CDATA[Why tail risk is often underpriced and common fraud metrics can be misleading]]></description><link>https://www.zhizhi-gewu.com/p/fraud-management-and-the-pricing</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/fraud-management-and-the-pricing</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 08 Mar 2026 00:59:43 GMT</pubDate><content:encoded><![CDATA[<p>As a new fraud manager, I started reflecting on the fundamental principles of fraud management. I began by looking into the metrics commonly used by fraud teams, such as precision, FPR, and recall. However, I soon realized how difficult it is to set KPIs in fraud because of a core paradox: <strong>a<a href="https://www.zhizhi-gewu.com/p/absence-of-evidence-evidence-of-absence">bsence of evidence is not evidence of absence</a></strong><a href="https://www.zhizhi-gewu.com/p/absence-of-evidence-evidence-of-absence">.</a></p><p>In this blog post, I want to go one step further and discuss the causes of this paradox, and why I believe it is extremely dangerous to optimize fraud management primarily around <a href="https://www.zhizhi-gewu.com/p/auc-ks-precision-and-recall-a-risk">precision, FPR, and recall</a>.</p><p><strong>Fraud events often have the following characteristics:</strong></p><ul><li><p>The frequency of fraud is small relative to the total number of events. At least, if the business is still surviving.</p></li><li><p>The loss from fraud is often many orders of magnitude larger than other types of loss, such as credit loss per event. A single fraud incident can wipe out a month&#8217;s profit.</p></li><li><p>The real damage usually comes from previously unknown attack vectors.</p></li><li><p>Fraud events happen more often than we think. When we say &#8220;low frequency,&#8221; we may imagine once every few years. In reality, they may happen much more often.</p></li></ul><p>The consequences are as follows.</p><p><strong>First Order Consequences</strong></p><ul><li><p>A 99.99% recall may still not be enough, because the remaining 0.01% of uncaptured fraud can still put the business at risk.</p></li><li><p>The absence of observed fraud events can create a false sense of security, even though only one disconfirming event is needed to reject the proposition that we are safe.</p></li><li><p>Thinking only in terms of frequency can put us in grave danger. First, we may assume fraud events are like 4-sigma events&#8212;rare and exceptional&#8212;when in fact they happen far more often, perhaps even every day. Second, even a truly rare event can still hurt the business badly because of the scale of exposure.</p></li><li><p>Fraud events cannot be treated in isolation, because the business may never fully recover from a large incident. There is path dependency.</p></li></ul><p><strong>Second Order Consequences</strong></p><ul><li><p>Focusing too much on precision, FPR, and recall is dangerous. These metrics can miss the small fraction of unknown risks that may cause losses 1000 times larger than what we are prepared for.</p></li><li><p>Similarly, machine learning models are not sufficient for dealing with extreme unknowns. Over-focusing on deploying more advanced models can shift attention away from what truly matters.</p></li><li><p>Average risk can be highly misleading. It changes dramatically once an outlier occurs, and those outliers are often both more frequent and much larger than people expect. Monitoring average risk alone can create a false sense of security.</p></li><li><p>Frequency-based caps, such as the number of transactions allowed, are conceptually incomplete because a single event can still wipe you out.</p></li></ul><p><strong>Strategy</strong></p><p>Are we doomed then? Not necessarily.</p><p>Although it is nearly impossible to predict or control exactly when fraud will happen, exposure can still be controlled. I would argue that the primary responsibility of a fraud team is to <strong>cap the downside of the business</strong>. In the process, the team should also aim to reduce the premium paid for that protection&#8212;for example, by lowering false positives. But uncapped exposure is non-negotiable.</p><p>A cap may look too simple, but we are not here to impress our peers.</p><p>Finally, fraud risk practitioners deserve more respect. In many ways, the fraud team is effectively the seller of a put option, while the business is the buyer. The fraud team&#8217;s upside is capped. The best outcome it can possibly deliver is close to 100% recall, which is impossible to achieve in practice. Yet its downside is theoretically uncapped. When an extreme event happens, the fraud team is often the one blamed.</p><p>At the same time, the premium paid for this protection&#8212;not in salary, but in lost business opportunities&#8212;is often mispriced relative to the guarantee the fraud team is expected to provide. The business tends to focus on frequency: fraud is unlikely, so why sacrifice business volume to manage a &#8220;long-tail&#8221; event? Fraud teams, by contrast, have to focus on magnitude.</p>]]></content:encoded></item><item><title><![CDATA[AI in My Daily Workflow: Use Cases and Reflections]]></title><description><![CDATA[The Last Human Job: Deciding What We Want]]></description><link>https://www.zhizhi-gewu.com/p/ai-in-my-daily-workflow-use-cases</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/ai-in-my-daily-workflow-use-cases</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Fri, 20 Feb 2026 06:17:33 GMT</pubDate><content:encoded><![CDATA[<p><strong>Use Cases</strong></p><p><strong>Understand a codebase quickly.</strong> <a href="https://deepwiki.com/">DeepWiki</a> provides a high-level, human-readable overview of any codebase &#8212; far more efficient than reading raw code. You can also ask questions directly about the project structure and logic.</p><p><strong>Automate workflows and solve problems computationally.</strong> Claude Code is my go-to here.</p><ul><li><p>Recently, I used Claude Code (with Agent Teams) to generate personal financial statements from bank, broker, and credit card statements. The result was surprisingly good. I spent about one working day polishing the output &#8212; something that would have taken me weeks to build from scratch.</p></li><li><p>Previously, I built a property listing monitor that pulls new listings from targeted developments and sends updates via Telegram. (<a href="https://github.com/kychanbp/propertyguru_listing_monitor">Github</a>)</p></li></ul><p><strong>Q&amp;A &#8212; information retrieval and brainstorming.</strong> I use Gemini across the board: text, images, and video (directly on YouTube). Frontier models are converging in capability, and Gemini offers better value for money. The bundled 2TB Google Drive is a bonus I always end up using anyway.</p><p><strong>Understand a new subject quickly.</strong> My workflow:</p><ol><li><p>Find resources via <a href="https://scholar.google.com/scholar_labs/search">Google Scholar Labs</a>, Gemini Deep Research, or Zhihu AI Chat (<a href="https://zhida.zhihu.com/">&#30693;&#20046;&#30452;&#31572;</a>).</p></li><li><p>Upload selected resources to NotebookLM and start asking questions.</p></li></ol><p>I recently used this to get up to speed on <a href="https://notebooklm.google.com/notebook/6c533dba-5d01-4c8c-a76e-193dc6e94e05">group fraud detection</a> as part of a push to take on broader antifraud responsibilities &#8212; what might have taken weeks of reading compressed into a few focused sessions.</p><p><strong>Build a personal knowledge base.</strong> I store personal notes in Markdown, use Claude Code to build a RAG server on top of them, and pipe in RSS feeds converted to Markdown. Everything becomes searchable and queryable.</p><div><hr></div><p><strong>Reflections</strong></p><p>AI has fundamentally changed how I learn. I spend far less time on Google Search &#8212; only when I need to trace a primary source. The structured summaries AI produces help me grasp the shape of a subject much faster than stitching together search results ever did.</p><p>AI is already very capable at coding. More importantly, coding is just a means to an end &#8212; a method for solving problems. AI lowers the barrier between having a problem and implementing a computational solution. If something <em>can</em> be solved programmatically, AI dramatically accelerates getting there.</p><p>One thing I&#8217;ve noticed with Claude Code specifically: memory and context have improved significantly. Previously, every new session started from scratch with no accumulated context. Now, the agent can automatically build a memory file or be explicitly prompted to maintain one across sessions. It feels more like working with a collaborator than restarting a tool.</p><p>That said, QA and critical thinking remain distinctly human strengths. AI output still needs to be sense-checked. Common sense matters &#8212; the ability to notice when something doesn&#8217;t add up, when a result feels off, when the logic is internally consistent but wrong in practice. Judgment about <em>what&#8217;s right</em> is still something humans do better.</p><p>Looking further ahead, once AI reliably solves coding, it gains the ability to improve itself. At that point, the barrier to completing any cognitive task effectively disappears. The human role shifts toward defining objectives &#8212; what we want the world to look like, what problems are worth solving, and what the company should be doing. AI handles the execution.</p><p>But there&#8217;s a second, subtler role: QA at the objective level. We&#8217;ll need to examine AI-proposed solutions and verify they actually align with what we intended &#8212; until AI develops its own objectives, and becomes sophisticated enough that the misalignment isn&#8217;t obvious. That&#8217;s the part worth thinking carefully about now.</p>]]></content:encoded></item><item><title><![CDATA[The Risk Metric Translation Layer: Why Precision and FPR Aren't Mirror Images]]></title><description><![CDATA[Sometimes, I found it inefficient to communicate the antifraud team's performance metrics, such as precision and false positive rate, to other teams or even within the antifraud team.]]></description><link>https://www.zhizhi-gewu.com/p/the-risk-metric-translation-layer</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/the-risk-metric-translation-layer</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 07 Feb 2026 08:10:00 GMT</pubDate><content:encoded><![CDATA[<p>Sometimes, I found it inefficient to communicate the antifraud team's performance metrics, such as precision and false positive rate, to other teams or even within the antifraud team. It is because those words are also used loosely in everyday communication. I said, &#8220;Our policy precision is 90%.&#8221; The counterpart replied, &#8220;10% false positive rate is too high to accept.&#8221; You know there is a gap in the understanding of precision and false positive rate (FPR). </p><p>On the business side, when they say &#8220;10% FPR is too high,&#8221; they mean: out of all the good users, how many are falsely labelled as bad? They worried about user experience. On the risk side, when we say &#8220;precision is 90%&#8221;, we mean: out of all the users who are being labelled as bad, how many are actually bad?. We are concerned about the performance of the models/rules themselves. By deriving the relationship, we can bridge the gaps between two metrics.</p><p>Let&#8217;s revisit the <a href="https://www.zhizhi-gewu.com/p/auc-ks-precision-and-recall-a-risk">two-room analogy</a> for intuitive understanding. FPR only cares about the good users&#8217; room (N). If FPR increases, the absolute number of false positives increases. Precision [TP/(TP+FP)]will drop as well because the denominator became larger. But let&#8217;s keep precision constant since we want to find the relationship between the two. This forces the model to capture more TP. In other words, recall (TP/P) must increase. Finally, since FPR and recall are just ratios, the relative number of N and P also affects the relationship, a.ka. prevalence [P/(P+N)].  To summarize, precision and FPR are connected by recall and prevalence. 90% precision does not imply an FPR of 10%, because we must also account for recall and the natural fraud rate (prevalence). </p><p>Mathematically, we can derive the relationship as:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;\\begin{align}\nprecision &amp;= \\frac{TP}{TP + FP} \\\\\n&amp;= \\frac{TP}{TP + N \\cdot FPR} \\\\\n&amp;= \\frac{P \\cdot Recall}{P \\cdot Recall + N \\cdot FPR} \\\\\n&amp;= \\frac{\\theta \\cdot Recall}{\\theta \\cdot Recall + (1-\\theta )\\cdot FPR}\n\\end{align}&quot;,&quot;id&quot;:&quot;GLFYRUITGP&quot;}" data-component-name="LatexBlockToDOM"></div><p>where theta is the prevalence.</p><p>Consider the following scenarios to consolidate the intuition, assuming the model is reasonably good (both precision and recall are high):</p><ul><li><p>FPR is low, and precision is high when prevalence is low.</p></li><li><p>FPR and precision are high when prevalence is high.</p></li></ul><p>You may already notice that even if model performance is poor, FPR can be low if prevalence is very low. For example, if the prevalence is 0.1%, the recall is 10%, the precision is 20%, and the FPR is only 0.04%.</p><p>Finally, the question is which number should we use. I think both should be reported separately, as the audiences are different. For the risk team, we should focus on precision and recall, as they are directly linked to model/policy performance. However, we should also check FPR as it affects the user experience.</p><p></p><p></p>]]></content:encoded></item><item><title><![CDATA[Absence of Evidence ≠ Evidence of Absence: Rethinking Fraud Prevention KPIs]]></title><description><![CDATA[The Zero-Incident Paradox: Why Silence Isn't Always Golden in Fraud Prevention Using Bayesian reasoning to measure what you cannot directly observe]]></description><link>https://www.zhizhi-gewu.com/p/absence-of-evidence-evidence-of-absence</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/absence-of-evidence-evidence-of-absence</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 24 Jan 2026 02:00:54 GMT</pubDate><content:encoded><![CDATA[<h2><strong>The Core Paradox</strong></h2><p>The Fundamental Challenge: Low incident rates in fraud prevention could indicate either:</p><ul><li><p><strong>Strong defenses</strong> - The system is working effectively</p></li><li><p><strong>Lack of attempts</strong> - There simply haven&#8217;t been many fraud attempts</p></li></ul><p>This ambiguity makes it difficult to assess the true effectiveness of fraud prevention measures.</p><p><strong>The Unknown Unknowns Problem</strong></p><p>Beyond measuring what we know, there&#8217;s a deeper challenge: <strong>you cannot directly measure what you are unaware of</strong>. This is the classic &#8220;unknown unknowns&#8221; dilemma in security and fraud prevention.</p><h2><strong>Bayesian Framework</strong></h2><p>To address these challenges, we can apply Bayesian reasoning:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;P(\\text{Fraud}|\\text{No Incident}) = \\frac{P(\\text{No Incident}|\\text{Fraud}) \\cdot P(\\text{Fraud})}{P(\\text{No Incident})}&quot;,&quot;id&quot;:&quot;JDKNFZKVVU&quot;}" data-component-name="LatexBlockToDOM"></div><p><strong>Interpreting the Evidence</strong></p><ul><li><p><strong>When the likelihood is high</strong>: <em>P</em>(No Incident&#8739;Fraud) is high</p><ul><li><p>The absence of fraud incidents strongly indicates effective fraud prevention</p></li></ul></li><li><p><strong>When the likelihood is low</strong>: <em>P</em>(No Incident&#8739;Fraud) is low</p><ul><li><p>The Absence of incidents provides <strong>little information</strong> about fraud prevention effectiveness</p></li></ul></li></ul><h2><strong>Measuring the Unmeasurable</strong></h2><h3><strong>The Challenge</strong></h3><p>Evaluating the likelihoods requires:</p><ul><li><p>Understanding the threat landscape</p></li><li><p>Assessing coverage of known fraud methods</p></li><li><p>Accounting for unknown fraud methods (which cannot be directly measured)</p></li></ul><h3><strong>Indirect Measurement Methods</strong></h3><p><strong>Approaches to Discover Unknown Unknowns</strong></p><p>When direct measurement is impossible, we can use these indirect methods:</p><ol><li><p><strong>Red Teaming</strong> - Simulate adversarial attacks to find vulnerabilities</p></li><li><p><strong>External Threat Intelligence</strong> - Learn from industry patterns and breaches</p></li><li><p><strong>Post-Mortem Analysis</strong> - Learn from failures when they occur</p></li><li><p><strong>First Principles Decomposition</strong> - Break down attack surfaces systematically</p></li></ol><h2><strong>Practical Framework</strong></h2><p>Measuring Fraud Prevention Success: To measure the success of fraud prevention in low incident scenarios, focus on:</p><ol><li><p><strong>Coverage Metrics</strong></p><ol><li><p>Percentage of known fraud methods with active defenses</p></li><li><p>Depth and breadth of detection capabilities (Cost of attack)</p></li></ol></li><li><p><strong>Discovery Rate</strong></p><ol><li><p>Incremental discovery of new fraud methods over time</p></li><li><p>Rate of vulnerability identification and remediation</p></li></ol></li><li><p><strong>Process Interception Metrics</strong></p><ol><li><p>Trigger rate: Proportion of applications/transactions flagged by rules</p></li><li><p>Hit rate: Proportion of triggered cases with confirmed abnormal behaviors (grey areas go to manual review)</p></li></ol></li></ol><p>This approach helps infer effectiveness even when direct incident data is sparse.</p>]]></content:encoded></item><item><title><![CDATA[Can You Solve This? Calculating KS from a Quarter-Circle ROC Curve]]></title><description><![CDATA[I recently came across an interesting interview question that bridges geometry and risk modeling: &#8220;If a model&#8217;s ROC curve is a perfect concave down quarter-circle, what is its KS value?&#8221;]]></description><link>https://www.zhizhi-gewu.com/p/can-you-solve-this-calculating-ks</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/can-you-solve-this-calculating-ks</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 11 Jan 2026 11:06:47 GMT</pubDate><content:encoded><![CDATA[<p>I recently came across an <a href="https://zhuanlan.zhihu.com/p/367772812">interesting interview question</a> that bridges geometry and risk modeling: &#8220;If a model&#8217;s ROC curve is a perfect concave down quarter-circle, what is its KS value?&#8221;</p><p>In my <a href="https://www.zhizhi-gewu.com/p/auc-ks-precision-and-recall-a-risk">last post</a>, I discussed the relationship between AUC and KS. Let&#8217;s build on that foundation to tackle this new question. Let&#8217;s recap the key concepts:</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><ul><li><p><strong>ROC Curve</strong>: A plot of the true positive rate against the false positive rate. A good model&#8217;s ROC curve bows towards the top-left corner because it maximizes true positives while minimizing false positives.</p></li><li><p><strong>KS Statistic</strong>: The maximum vertical distance between the cumulative distribution functions (CDFs) of the positive and negative classes. It quantifies how well the model separates the two classes.</p></li></ul><p>The <strong>key insight</strong> to solving this problem lies in the fact that the true positive rate is the same as the cumulative distribution function (CDF) for the positive class, and the false positive rate is the CDF for the negative class. Therefore, the KS statistic can be expressed as: <em>KS</em>=max(<em>TPR</em>&#8722;<em>FPR</em>). (Please refer to <a href="https://www.zhizhi-gewu.com/p/auc-ks-precision-and-recall-a-risk">my previous post</a> for a detailed explanation of this relationship.)</p><p>The equation for a circle centered at (<em>h</em>,<em>k</em>) with radius <em>r</em> is: (x-h)^2 + (y-k)^2 = r^2. For a quarter-circle ROC curve centered at (1,0) with radius 1, the equation simplifies to: (x-1)^2 + y^2 = 1. Rearranging gives us: </p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;y = \\sqrt{1 - (x - 1)^2} = \\sqrt{2x - x^2}&quot;,&quot;id&quot;:&quot;JLJEUQVHYW&quot;}" data-component-name="LatexBlockToDOM"></div><p>To find the KS value, we need to maximize y&#8722;x<em>y</em>&#8722;<em>x</em> over the interval [0,1]. This leads us to the function: </p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;f(x) = \\sqrt{2x - x^2} - x&quot;,&quot;id&quot;:&quot;OKEODIUJBK&quot;}" data-component-name="LatexBlockToDOM"></div><p>. Taking the derivative and setting it to zero gives us the critical points:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;f'(x) = \\frac{1 - x}{\\sqrt{2x - x^2}} - 1 = 0&quot;,&quot;id&quot;:&quot;JHZDOPPKWI&quot;}" data-component-name="LatexBlockToDOM"></div><p>Solving this equation, we find: </p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;x = 1 - \\frac{\\sqrt{2}}{2}&quot;,&quot;id&quot;:&quot;CAIQSYZYXC&quot;}" data-component-name="LatexBlockToDOM"></div><p> (Discarded another solution because the range of <em>x</em> is [0,1]).</p><p>Plugging this back into our function, we calculate:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;KS = f(x) = \\frac{\\sqrt{2}}{2} - \\left(1 - \\frac{\\sqrt{2}}{2}\\right) = \\sqrt{2} - 1 \\approx 0.414&quot;,&quot;id&quot;:&quot;RDZCJGMQDN&quot;}" data-component-name="LatexBlockToDOM"></div><p>If you can follow this reasoning and arrive at the same conclusion, congratulations! You&#8217;ve successfully connected geometric intuition with statistical modeling. The key learning of the question is that KS alone does not fully capture model performance. It only anchors the shape of the ROC curve.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Same Math, Different Enemies: The Unit Economics of Credit and Fraud Risk]]></title><description><![CDATA[In my previous post, I discussed standard risk metrics like AUC, KS, Precision, and Recall.]]></description><link>https://www.zhizhi-gewu.com/p/same-math-different-enemies-the-unit</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/same-math-different-enemies-the-unit</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 28 Dec 2025 02:00:21 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/ac3950d7-3d29-4728-ad21-9893f94e9745_2816x1536.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In my previous post, I discussed standard risk metrics like AUC, KS, Precision, and Recall. But metrics are just a means to an end. How do the actual <em>decision-making</em> processes differ between Credit Risk and Anti-Fraud?</p><p>Having recently rotated from the Credit Risk team to the Anti-Fraud team, I&#8217;ve realized that while the terminology differs, the fundamental logic&#8212;derived from the first principle of <strong>Maximizing Profit</strong>&#8212;is remarkably similar. Below, I demonstrate the mathematical unity of the two domains and highlight the key strategic differences.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h2><strong>The Shared Math: Maximizing Incremental Value</strong></h2><p>Let&#8217;s start with the basics. Total profit equals the revenue from good users minus the losses from bad users.</p><ul><li><p><strong>True Negatives (TN):</strong> Approved users who pay (Profit).</p></li><li><p><strong>False Negatives (FN):</strong> Approved users who default/fraud (Loss).</p></li></ul><p>Define positive as &#8220;we block/reject because predicted bad,&#8221; and negative as &#8220;we approve.</p><p>We can express this as:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;Total\\ Profit = TN \\cdot R - FN \\cdot L&quot;,&quot;id&quot;:&quot;VVSJKWWODH&quot;}" data-component-name="LatexBlockToDOM"></div><p><em>Where </em>R<em> is Revenue per Good User and </em>L<em> is Loss per Bad User.</em></p><p>However, this equation is too static for strategy design. To decide whether to approve or reject a specific segment, we need to look at the <strong>Counterfactual</strong>: <em>What is the value added by our decision model compared to doing nothing?</em></p><p>If we compare our strategy against a baseline of &#8220;Approving Everyone,&#8221; the Total Profit can be rewritten as:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;\\begin{align*}\nTotal\\ Profit &amp;= TN \\cdot R - FN \\cdot L \\\\\n&amp;= (N_{good} - FP) \\cdot R - (N_{bad} - TP) \\cdot L \\\\\n&amp;= \\underbrace{(N_{good} \\cdot R - N_{bad} \\cdot L)}_{\\text{Baseline Profit}} + \\underbrace{(TP \\cdot L - FP \\cdot R)}_{\\text{Incremental Strategy Value}}\n\\end{align*}&quot;,&quot;id&quot;:&quot;NEVNJOAVAL&quot;}" data-component-name="LatexBlockToDOM"></div><ul><li><p>N_good,N_bad: Total population of good and bad users (Constant).</p></li><li><p>TP&#8901;L: The loss <strong>saved</strong> by correctly blocking bad users.</p></li><li><p>FP&#8901;R: The revenue <strong>sacrificed</strong> by incorrectly blocking good users.</p></li></ul><p>Since the Baseline Profit is constant (determined by the population), maximizing Total Profit is mathematically identical to maximizing <strong>Incremental Strategy Value</strong>.</p><h2><strong>The Optimality Condition: MC = MB</strong></h2><p>Economics teaches us that profit is maximized when <strong>Marginal Cost equals Marginal Benefit</strong>.</p><p>Let p^ be the probability that the next user we assess is &#8220;Bad.&#8221;</p><ul><li><p><strong>Marginal Benefit:</strong> If we block them and they are Bad, we save L.</p></li><li><p><strong>Marginal Cost:</strong> If we block them and they are Good, we lose R.</p></li></ul><p>We should stop blocking exactly when the cost outweighs the benefit:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;\\begin{align*}\nMarginal\\ Benefit - Marginal\\ Cost &amp;= 0 \\\\\n\\hat{p} \\cdot L - (1 - \\hat{p}) \\cdot R &amp;= 0 \\\\\n\\hat{p} \\cdot (L + R) &amp;= R \\\\\n\\hat{p} &amp;= \\frac{R}{L + R}\n\\end{align*}&quot;,&quot;id&quot;:&quot;ERVYDVPNMQ&quot;}" data-component-name="LatexBlockToDOM"></div><p>&#8203;&#8203;</p><p>This simple ratio, R/(L+R)&#8203;, is the &#8220;Universal Cutoff.&#8221; But the two teams view it from opposite ends.</p><h3><strong>1. The Anti-Fraud View (The Defender)</strong></h3><p>In fraud, we &#8220;block&#8221; transactions.</p><ul><li><p><strong>Revenue (R):</strong> The cost of insulting a good customer (C_insult), which is the interest loss.</p></li><li><p><strong>Loss (L):</strong> The fraud loss saved (C_fraud), which is the principal loss.</p></li></ul><p>The probability p^here represents the probability of fraud&#8212;which is exactly the definition of <strong>Precision</strong> for that marginal alert.</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;Precision_{min} = \\frac{C_{insult}}{C_{insult} + C_{fraud}} = \\frac{Interest}{Interest + Principal}&quot;,&quot;id&quot;:&quot;VCXMFWSFNC&quot;}" data-component-name="LatexBlockToDOM"></div><p>&#8203;&#8203;If the model&#8217;s precision drops below this threshold, the cost of insulting good customers exceeds the value of the fraud we stop.</p><h4><strong>A Note on Terminology: Why does Probability (p^) equal Precision?</strong></h4><p>It might seem confusing to equate a single user&#8217;s probability score (p^&#8203;) with a group statistic like Precision. However, at the margin, they are identical. Simply apply the Law of Large Numbers:</p><p>Imagine our model assigns a risk score of <strong>0.20</strong> to a specific transaction. This prediction literally means: <em>&#8220;There is a 20% chance this specific transaction is fraud.&#8221;</em> , assuming the model is well-calibrated.</p><p>Now, imagine we gather <strong>100 transactions</strong> that all share this exact risk score of 0.20.</p><ul><li><p><strong>Expected Fraud (TP):</strong> 20</p></li><li><p><strong>Expected Legitimate (FP):</strong> 80</p></li></ul><p>If we block this specific bucket of users, the Precision is:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;Precision = \\frac{TP}{TP + FP} = \\frac{20}{20 + 80} = 20\\%&quot;,&quot;id&quot;:&quot;UNRJTFTAVO&quot;}" data-component-name="LatexBlockToDOM"></div><p>Thus, the <strong>Probability Score</strong> at the cutoff is simply the <strong>Marginal Precision</strong> of the decision at that cutoff.</p><h3><strong>2. The Credit Risk View (The Attacker)</strong></h3><p>In credit, we &#8220;approve&#8221; loans.</p><ul><li><p><strong>Revenue (R):</strong> Interest Income.</p></li><li><p><strong>Loss (L):</strong> Principal Loss.</p></li></ul><p>The probability p^&#8203; here represents the <strong>Probability of Default (PD)</strong>.</p><p></p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;PD_{max} = \\frac{Interest}{Principal + Interest}&quot;,&quot;id&quot;:&quot;YZKABCXXAE&quot;}" data-component-name="LatexBlockToDOM"></div><p>&#8203;If the user&#8217;s PD rises above this threshold, the expected principal loss exceeds the potential interest income. (Of course, this is a simplified way to calculate revenue and cost but it captures the essence.)</p><h3><strong>Conclusion: Same Line, Opposite Directions</strong></h3><p>Mathematically, <strong>Minimum Precision</strong> and <strong>Maximum PD</strong> are the exact same number.</p><ul><li><p>The <strong>Anti-Fraud</strong> team defends the gate, blocking bad actors until the precision drops to the cutoff.</p></li><li><p>The <strong>Credit Risk</strong> team expands the market, approving users until the risk rises to the cutoff.</p></li></ul><h2><strong>The Strategic Divergence: Different Enemies</strong></h2><p>If the math is identical, why do the jobs feel so different? Because while the <em>equation</em> is the same, the <em>enemy</em> is not.</p><h3><strong>1. Definition of &#8220;Bad&#8221;</strong></h3><ul><li><p><strong>Credit Risk:</strong> &#8220;Bad&#8221; is defined solely by default.</p></li><li><p><strong>Anti-Fraud:</strong> &#8220;Bad&#8221; is defined by intent (Deception). Thus, default alone is not sufficient. The team will also look for suspicous patterns.</p></li></ul><h3><strong>2. Static vs. Dynamic (Game Theory)</strong></h3><p>This is the most critical difference.</p><ul><li><p><strong>Credit Risk mostly plays against Nature.</strong> Borrowers are relatively stable. A user with a 620 credit score today behaves similarly to a 620 user yesterday. Historical &#8220;Vintage&#8221; data is highly predictive of the future.</p></li><li><p><strong>Anti-Fraud plays against an Adversary.</strong> Fraudsters are intelligent, coordinated, and reactive. If you set a static rule to block X, they immediately shift to Y.</p><ul><li><p><em>Implication:</em> Credit teams optimize for <strong>Efficiency</strong> (Calibration). Fraud teams must optimize for <strong>Adaptability</strong> (Exploration).</p></li></ul></li></ul><h3><strong>3. The Action Space</strong></h3><ul><li><p><strong>Anti-Fraud:</strong> The decision is usually binary (Approve vs. Reject) or friction-based (Step-up Verification).</p></li><li><p><strong>Credit Risk:</strong> The decision is multi-dimensional. We can manage risk not just by rejecting, but by adjusting the <strong>Credit Limit</strong>, <strong>Tenure</strong>, or <strong>Pricing (EIR)</strong>. We have more levers to force the unit economics to work.</p></li></ul><h3><strong>4. The Entity Dimension (Multi-Modal Risk)</strong></h3><ul><li><p><strong>Credit Risk</strong> is almost exclusively <strong>User-Centric</strong>. We underwrite the person (or the business entity) applying for funds. The unit of analysis is stable.</p></li><li><p><strong>Anti-Fraud</strong> is <strong>Multi-Modal</strong>. We don&#8217;t just assess the User; we assess the Device, the IP address, the Credit Card, and the Merchant.</p></li></ul><p><strong>The Economic Implication:</strong> The variables in our profit equation (C_insult&#8203; and C_fraud&#8203;) shift drastically depending on <em>what</em> we are blocking.</p><ul><li><p><strong>Blocking a Device:</strong> If I block a suspicious device ID, the Cinsult<em>Cinsult</em>&#8203; might be low (the user is annoyed but can switch devices). <em>Result:</em> I can afford a <strong>lower precision</strong> threshold.</p></li><li><p><strong>Blocking a Merchant:</strong> If I block a merchant in a marketplace, I cut off revenue from <em>all</em> their customers. The C_insult(Lost Revenue) is massive. <em>Result:</em> I need an extremely <strong>high precision</strong> threshold&#8212;often requiring manual review&#8212;before pulling the trigger.</p></li></ul><p>While Credit Risk optimizes one curve (User Risk), Anti-Fraud constantly juggles multiple curves with different breakeven points.</p><h2>Closing Thoughts</h2><p>I am still very much a learner in this space, but realizing that Credit Risk and Anti-Fraud share the same Unit Economics has been a helpful anchor for me. </p><p>It means that while I am learning new tactics (Game Theory, Pattern Recognition), the underlying grammar of <strong>Profit Maximization</strong> remains the same. </p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[AUC, KS, Precision, and Recall: A Risk Analyst’s Guide]]></title><description><![CDATA[As analysts in the risk management industry, we live and breathe acronyms like AUC (Area Under the Curve) and KS (Kolmogorov-Smirnov).]]></description><link>https://www.zhizhi-gewu.com/p/auc-ks-precision-and-recall-a-risk</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/auc-ks-precision-and-recall-a-risk</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 20 Dec 2025 02:00:41 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/00e17c0d-0bf8-43e5-8b80-eef275a74334_1174x781.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>As analysts in the risk management industry, we live and breathe acronyms like <strong>AUC</strong> (Area Under the Curve) and <strong>KS</strong> (Kolmogorov-Smirnov). These are the gold standards for evaluating classification models in credit risk. However, if you&#8212;like me&#8212;have recently rotated into an <strong>Anti-Fraud</strong> team from credit risk team, you&#8217;ve likely encountered two different metric kings: <strong>Precision</strong> and <strong>Recall</strong>.</p><p>How do these metrics relate to each other?</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h2><strong>The &#8220;Two Rooms&#8221; Analogy</strong></h2><p>The standard confusion matrix terms&#8212;True Positive (TP), False Positive (FP), True Negative (TN), and False Negative (FN)&#8212;can be headache-inducing to communicate in practice.</p><p>To simplify this, imagine two separate rooms:</p><ol><li><p><strong>The Bad Room:</strong> Contains <em>only</em> actual bad actors (Total Bad = TP + FN).</p></li><li><p><strong>The Good Room:</strong> Contains <em>only</em> actual good users (Total Good = TN + FP).</p></li></ol><p>Your model is a gatekeeper. You walk into each room with your classifier.</p><ul><li><p>In the <strong>Bad Room</strong>, you want the model to identify everyone as bad. The percentage of people you successfully catch here is your <strong>True Positive Rate (TPR)</strong>.</p></li><li><p>In the <strong>Good Room</strong>, you want the model to identify no one as bad. The percentage of people you mistakenly flag as bad here is your <strong>False Positive Rate (FPR)</strong>.</p></li></ul><p>A perfect model has a TPR of 100% (catches everyone in the Bad Room) and an FPR of 0% (flags no one in the Good Room).</p><h3><strong>Visualizing AUC and KS</strong></h3><p>Realistically, models output a <em>probability</em> (e.g., &#8220;There is a 90% chance this user is a fraud&#8221;). We use a &#8220;threshold dial&#8221; to decide who to flag.</p><p>Imagine turning this dial:</p><ul><li><p><strong>Threshold 100% (Strict):</strong> You only flag apparent fraud. You catch almost no one in the Bad Room (TPR&#8776;0), but you also annoy no one in the Good Room (FPR&#8776;0). This is the point (0,0)on the ROC plot.</p></li><li><p><strong>Threshold 0% (Loose):</strong> You flag everyone. You catch every fraudster (TPR=100%), but you also falsely flag every good user (FPR=100%). This is the point (1,1).</p></li></ul><p><strong>AUC (Area Under the Curve)</strong> measures the model&#8217;s performance across <em>all possible settings</em> of this dial. It plots TPR against FPR. A random guess gives you a straight diagonal line (AUC 0.5), meaning for every extra bad person you catch, you annoy a proportional number of good people. A good model bows upward, maximizing the gap between the TPR and FPR.</p><p><strong>KS (Kolmogorov-Smirnov)</strong> focuses on the single best point on that curve. It is simply the <strong>maximum difference</strong> between TPR and FPR. While AUC looks at the whole story, KS asks: &#8220;At the single best setting of the dial, how much separation can we get between the good and bad populations?&#8221; </p><h4>Deep Dive: Why KS is Cumulative</h4><p>KS is usually plotted using the cumulative number of bads and goods. The maximum difference between the two curves is the KS.</p><p>Imagine taking all the people from <em>both</em> rooms and lining them up in a single queue, sorted by their model score from <strong>Highest (Most Suspicious)</strong> to <strong>Lowest (Least Suspicious)</strong>.</p><p>Now, imagine walking down this line from the start. This is equivalent to lowering your threshold.</p><ol><li><p><strong>Cumulative Bad (TPR):</strong> Every time you walk past a <em>Bad Person</em>, you add them to your count. If there are 100 Bad People total, and you have passed 50 of them, your &#8220;Cumulative Bad %&#8221; is 50%. <strong>This is exactly the True Positive Rate (TPR).</strong></p></li><li><p><strong>Cumulative Good (FPR):</strong> Every time you walk past a <em>Good Person</em>, you add them to your count. If there are 1,000 Good People total, and you have passed 100 of them, your &#8220;Cumulative Good %&#8221; is 10%. <strong>This is exactly the False Positive Rate (FPR)</strong></p></li></ol><p>Below is a simulation dashboard I asked Gemini to build to visualize these concepts: <a href="https://gemini.google.com/share/0ade30dc6f52">Link</a></p><h3><strong>Why Anti-Fraud Cares: Precision and Recall</strong></h3><p>In the anti-fraud world, <strong>Recall</strong> is just another word for TPR (Bad Room coverage). But <strong>Precision</strong> is different.</p><p>While TPR and FPR require you to look at the rooms separately, <strong>Precision</strong> requires you to look at <em>who the model flagged</em> from both rooms combined. It asks: <em>&#8220;Of all the people the model claimed were bad, how many were actually bad?&#8221;</em></p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;Precision = \\frac{TP}{TP + FP}&quot;,&quot;id&quot;:&quot;UMOMZMZBHF&quot;}" data-component-name="LatexBlockToDOM"></div><p>&#8203;Why is this preferred over FPR in fraud operations?</p><ol><li><p><strong>Operational Reality:</strong> In fraud, a positive flag usually triggers a manual review, an SMS alert, or a transaction block. These actions have direct costs (agent time) and customer friction (insult). Precision measures the &#8220;purity&#8221; of the alert queue.</p></li><li><p><strong>The Class Imbalance Problem:</strong> This is the key differentiator. Precision is highly sensitive to the <strong>Bad Rate</strong>.</p><ul><li><p><strong>FPR</strong> is calculated <em>only</em> inside the Good Room. If you double the number of good customers, the FPR remains stable.</p></li><li><p><strong>Precision</strong> depends on the ratio of good to bad. If the number of good customers explodes while the number of fraudsters stays the same, your Precision will plummet because the &#8220;noise&#8221; (False Positives) drowns out the signal.</p></li></ul></li></ol><p>In summary, the Modeling Team often focuses on <strong>AUC/KS</strong> because they measure the model&#8217;s pure ability to rank order, independent of the portfolio&#8217;s bad rate. Anti-Fraud focuses on <strong>Precision</strong> because it reflects the actual operational pain of sifting through false alarms in a sea of good transactions.</p><h2><strong>Why does the credit risk team seldom look into Precision and Recall?</strong></h2><p>This question deserves a dedicated deep dive in the next post.</p><p>Fundamentally, both Underwriting and Anti-Fraud teams share the exact same goal: maximizing profit. They simply approach the P&amp;L equation from opposite ends (Alert: Fraud is a dynamic process. The environment is not stable because the fraudster evolves.):</p><ul><li><p>Underwriting aims to maximize disbursements. They expand approvals until the marginal cost of defaults outweighs the marginal revenue from interest. Their constraint is the Breakeven Cost of Risk.</p></li><li><p>Anti-Fraud aims to minimize losses. They expand fraud detection until the cost of friction (insulting good customers) outweighs the savings from stopping fraud. Their constraint is Breakeven Precision.</p></li></ul><p>In my following note, I plan to demonstrate mathematically that these two concepts are identical: the Breakeven Cost of Risk in lending equals the Breakeven Precision in fraud prevention.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[It’s Not the Chaos, It’s the Expectation: A Framework for Deconstructing Anxiety]]></title><description><![CDATA[Recently, I&#8217;ve spent time reflecting on the problem of anxiety.]]></description><link>https://www.zhizhi-gewu.com/p/its-not-the-chaos-its-the-expectation</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/its-not-the-chaos-its-the-expectation</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 13 Dec 2025 01:00:45 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/8e786947-04d0-404c-a2e0-3dc55eaf7803_2816x1536.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Recently, I&#8217;ve spent time reflecting on the problem of anxiety. It&#8217;s something many of us deal with, often feeling like a vague, overwhelming fog.</p><p>But when I sat down to analyze my own experiences&#8212;trying to pinpoint exactly <em>where</em> that feeling comes from&#8212;I realized that anxiety isn&#8217;t usually random chaos. It&#8217;s almost always structural. It stems from specific flaws in the mental models I use to navigate the world.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>If we can identify the roots of the anxiety, we can build a framework to handle it.</p><p>Through my reflection, I identified three main sources of anxiety, how they interconnect, and a two-step approach to regain our footing.</p><h2>Part 1: The Diagnosis (Where Anxiety Comes From)</h2><p>In my experience, anxiety isn&#8217;t usually caused by the event itself, but rather my <em>relationship</em> to the event. It almost always stems from one of these three situations:</p><h3>1. The Expectation Gap (The Illusion of Control)</h3><p>We often hear that anxiety comes from a &#8220;loss of control.&#8221; But that&#8217;s only half the story. Before you can lose control, there must be an <strong>expectation</strong> that you had it in the first place.</p><p>Anxiety thrives in the gap between our expectations and reality. Sometimes, we set the bar impossibly high, wanting control over things that are inherently uncontrollable. When we expect to be in the driver&#8217;s seat and reality suddenly grabs the wheel, panic sets in. The anxiety isn&#8217;t just about what is happening; it is the friction caused by our resistance to reality.</p><h3>2. The Value Vacuum (Not Knowing What You Want)</h3><p>If you don&#8217;t have a clear internal hierarchy of values&#8212;knowing exactly what you want and what is truly important to you&#8212;you become a reactive vessel for external pressures.</p><p>We live in an era of massive information flow and endless options. Without a strong internal compass, everything feels equally important. The urgent drowns out the important. This leads to chronic overwhelm as we try to juggle conflicting priorities that we didn&#8217;t even choose for ourselves.</p><h3>3. The Social Mirror (External Validation)</h3><p>This is perhaps the most paralyzing source: putting too much emphasis on other people&#8217;s views of us.</p><p>When we don&#8217;t know our own value (point #2), we outsource the measuring of it to society. We look into the &#8220;social mirror&#8221; to see if we are okay. The problem is that the mirror is constantly changing, and we have zero control over what others think. Basing your stability on something unstable is a recipe for constant anxiety.</p><h2>Part 2: The Prescription (How to Handle It)</h2><p>So, how do we break this cycle? The solution requires flipping our operating system. Instead of looking outward for cues, we must start inward.</p><h3>Step 1: Define Your &#8220;Self&#8221; First (First Principles Thinking)</h3><p>To handle anxiety, you must make your own self the priority. Before you look at the world, you have to look in the mirror and ask: <em>What do I actually want? What is genuinely important to me?</em></p><p>This is difficult. We are conditioned by social norms and existing incentive systems. We often treat these systems (like career ladders or social expectations) as rigid boundaries.</p><p>But we must remember that these systems were designed by <em>other people</em> to put constraints on behavior. Don&#8217;t treat them as immutable laws of physics before you even know what you want.</p><p><strong>The shift:</strong> Figure out what you value first. <em>After</em> you know what you want, then you can look at the societal constraints and decide consciously whether you want to work within them or break them. You become the actor, not the reactor.</p><h3>Step 2: The Spheres of Control</h3><p>Once you know what you want, the final step is calibrating your effort. While the Stoics famously spoke of the &#8220;dichotomy of control&#8221; (what is yours vs. what isn&#8217;t), I find it more useful to split control into three distinct rings.</p><p>Ring 1: Direct Control</p><p>This is your internal territory. It includes your actions, your effort, what you say, and how you allocate your time. This is the only place where you have total agency.</p><p>Ring 2: Influence</p><p>This is the gray area where most anxiety lives. It includes relationships, team decisions, negotiations, and probabilities. You can affect the outcome here, but you cannot dictate it.</p><p>Ring 3: No Control</p><p>This is the environment. It includes the macro economy, the weather, the past, and other people&#8217;s internal states.</p><p>How this fixes anxiety:</p><p>Anxiety is usually a result of trying to apply Ring 1 energy to a Ring 3 problem. To reduce it, you must map your worries to the correct ring:</p><ul><li><p><strong>In Ring 1:</strong> Act strongly. Focus on high-quality input and effort.</p></li><li><p><strong>In Ring 2:</strong> Experiment and negotiate. Influence the odds, but detach from the guarantee.</p></li><li><p><strong>In Ring 3:</strong> Practice acceptance. Observe it without trying to change it.</p></li></ul><h2>Moving Forward</h2><p>Anxiety often feels like a defect, but I&#8217;m starting to view it as a signal.</p><p>It&#8217;s a signal that my expectations are out of alignment with reality, or that I&#8217;m trying to control the uncontrollable. By using this framework to diagnose the source and placing my efforts in the correct ring, the fog begins to lift.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[The Art of Problem Solving: Defining the Gap and Trusting the Process]]></title><description><![CDATA[Do you have friends who seem naturally better at problem-solving?]]></description><link>https://www.zhizhi-gewu.com/p/the-art-of-problem-solving-defining</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/the-art-of-problem-solving-defining</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 23 Nov 2025 03:26:28 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/7931a49f-841f-409f-ae5e-94139c40b39a_2816x1536.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Do you have friends who seem naturally better at problem-solving? They possess sharper intuition and judgment. They appear smart and decisive, and in hindsight, they always seem to take the right steps. How can we emulate them and become better problem solvers ourselves?</p><p>We often say things like, &#8220;Meetings occupy too much of my time,&#8221; &#8220;We need a dashboard,&#8221; or &#8220;Sales are down.&#8221; We call these &#8220;problems.&#8221; But symptoms, solutions, tasks, or wishes are not problems. These are merely our observations or reactions. We cannot be better problem solvers if we cannot clearly define what a problem actually is.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>A problem is simply the distance between reality and expectation&#8212;the gap between the current state and the desired state. There are frameworks to help us define this, such as the CRIC framework (Context, Reality, Ideal, Consequences) or the 5-factor framework (Context, Criteria, Boundaries, Constraints, Stakeholders). Ultimately, to define a problem, we just need to ask: What is expected? Where are we now? And what stops us from moving from here to there?</p><p>A good problem-solver not only defines the problem clearly but also solves it. Since a problem is the gap between reality and expectation, good problem-solvers are often high achievers because they possess the means to get what they desire. Solving a problem is, at its heart, a decision-making process.</p><p>While discussing decision-making could fill another post, there are a few common pitfalls worth mentioning. First, do not judge a decision solely by its outcome. Uncertainty is inevitable; even a great decision-maker isn&#8217;t right every time. While you can look at their track record, remember that good decision-makers are often assigned the most difficult, uncertain tasks. It is easier said than done to separate process from outcome&#8212;especially when reward systems are outcome-based&#8212;but you must evaluate your decisions based on the process used to make them. This is why I write down my major decisions: so I can revisit the logic later, regardless of the result.</p><p>Another common error is &#8220;analysis paralysis&#8221;&#8212;indefinitely collecting information without acting. We must ask: Will knowing this extra piece of information change the decision? If the answer is no, we shouldn&#8217;t spend time or money acquiring it. Also, remember that decisions aren&#8217;t always binary (do it vs. don&#8217;t do it); they can be continuous, like deciding how much to invest.</p><p>Finally, a poor problem solver fails to consider the full range of outcomes. They do not treat decision-making as a matter of expected value. No action guarantees a specific result; there is always a probability of an undesired outcome. Estimating these probabilities requires experience, but the more you try to calibrate your estimates, the accurate you will be. You can even leverage existing data to build a model as a starting point.</p><p>Although natural intelligence plays a role, I believe problem-solving is a skill that can be improved through awareness and training. By doing so, we can raise the lower limit of our capability&#8212;assuming we agree that capability is a range, not a fixed number.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[My Singapore Property Purchase Journey (Resale Condo Edition)]]></title><description><![CDATA[A Data-Driven, Step-by-Step Guide to Navigating the Singapore Resale Condo Journey &#8212; from Budgeting and Valuation to Negotiation and Completion]]></description><link>https://www.zhizhi-gewu.com/p/my-singapore-property-purchase-journey</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/my-singapore-property-purchase-journey</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 25 Oct 2025 15:23:47 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/5fa96d15-9e77-4e9b-8ee8-7fc169be0822_1024x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p></p><p>Buying a home in Singapore is equal parts data exercise, emotional commitment, and an administrative obstacle course. This post documents my experience purchasing a <strong>resale condominium</strong> &#8212; from budgeting and deal sourcing to valuation, negotiation, and completion. The goal is to make the process less mysterious for anyone planning a similar journey.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><div><hr></div><h2>1. Typical Milestones</h2><p>The resale-condo journey can be divided into three major milestones:</p><p><strong>(a)</strong> <strong>Budgeting and Affordability</strong> &#8211; <em>What can I truly afford?</em><br><strong>(b)</strong> <strong>Property Deal Sourcing</strong> &#8211; <em>What do I actually want, and what is it worth?</em><br><strong>(c)</strong> <strong>Making a Purchase</strong> &#8211; <em>How do I manage the legal process and cash flow?</em></p><p>Thinking in these stages keeps the process structured and grounded in data rather than emotion.</p><h3>(a) Budgeting and Affordability</h3><p>The first step is to understand that a property&#8217;s cost goes far beyond the price tag. Buying a condo includes stamp duties, legal fees, and sometimes renovation costs &#8212; some payable via CPF, others strictly in cash. Knowing these distinctions early prevents liquidity shocks.</p><p>Equally critical is finding out <strong>what the bank thinks you can afford</strong>, not what you hope to. Getting an <strong>In-Principle Approval (IPA)</strong> from several banks translates your income, debt, and age into an actual loan ceiling. It&#8217;s the single number that defines your real search boundary.</p><h3>(b) Property Deal Sourcing</h3><p>Once your budget is clear, define what you need versus what you merely want. Listing platforms can make every condo look irresistible; your written criteria filter fantasy from reality.</p><p>Data keeps emotion in check. Compare recent transactions by PSF, lease start year, layout, and noise exposure. Look for fair value, not just pretty interiors.</p><h3>(c) Making a Purchase</h3><p>Finally, understand the <strong>timeline of legal steps and cash outflows</strong>. Each stage &#8212; option fee, exercise fee, stamp duties, and completion &#8212; happens on fixed dates. Planning this sequence ensures you always have funds ready when needed.</p><div><hr></div><h2>2. Budgeting and Affordability</h2><p>Date: 2025-10. Numbers may be outdated later.</p><p>Property ownership is a sequence of financial commitments that extends long after key collection. I classify them as <strong>acquisition</strong>, <strong>holding</strong>, and <strong>exit</strong> costs.</p><p>Building a simple <strong>Excel sheet</strong> helps you see the big picture &#8212; what can be paid by CPF, what needs cash, and when payments fall due. It also lets you test different price or loan scenarios.</p><h3>Acquisition Costs &#8211; Getting the Keys</h3><ul><li><p><strong>Purchase Price &amp; Downpayment.</strong> At least 25 % of the purchase price must come from your own funds (5 % in cash, 20 % in CPF or cash).</p></li><li><p><strong>Buyer&#8217;s Stamp Duty (BSD).</strong> Progressive tax on the higher of price or valuation.</p></li><li><p><strong>Additional Buyer&#8217;s Stamp Duty (ABSD).</strong> Applies to foreign or multiple-property buyers.</p></li><li><p><strong>Legal and Administrative Fees.</strong> Lawyer conveyancing, valuation reports, and miscellaneous filings.</p></li><li><p><strong>Initial Capital Expenditure.</strong> Renovation, furnishing, and minor repairs &#8212; often 5&#8211;10 % of price.</p></li></ul><p>An <strong>In-Principle Approval (IPA)</strong> anchors your expectations. It defines your borrowing limit and monthly repayment under Singapore&#8217;s Total Debt Servicing Ratio (TDSR), which caps total monthly debt at 55 % of gross income.</p><h3>Holding Costs &#8211; The Price of Ownership</h3><ul><li><p><strong>Financing Costs.</strong> Loan repayments vary with interest rates. Even a 0.5 % change can shift cash flow noticeably.</p></li><li><p><strong>MCST Fees.</strong> Management Corporation Strata Title fees fund common-area maintenance; these vary by facilities and unit size.</p></li><li><p><strong>Property Tax.</strong> Based on annual value and owner-occupation status; higher for investment properties.</p></li><li><p><strong>Maintenance &amp; Insurance.</strong> Routine servicing, repairs, and both fire and contents insurance.</p></li></ul><h3>Exit Costs &#8211; When You Sell or Refinance</h3><ul><li><p><strong>Seller&#8217;s Stamp Duty (SSD).</strong> Charged if you sell within four years; rates fall from 16 % to zero.</p></li><li><p><strong>Agent Commission.</strong> Typically 2 % + GST, negotiable for high-value units.</p></li><li><p><strong>Legal Fees &amp; Loan Penalties.</strong> Cover conveyancing for sale and any early-redemption fees.</p></li></ul><div><hr></div><h2>3. Property Deal Sourcing</h2><h3>Where to Find Listings</h3><p>The most active marketplaces are <strong><a href="https://www.propertyguru.com.sg/">PropertyGuru</a></strong> and <strong><a href="https://www.99.co/">99.co</a></strong> &#8212; good for browsing and gauging seller expectations.</p><p>For data-driven searches, <strong><a href="https://www.realsmart.sg/">RealSmart.sg</a></strong> and <strong><a href="https://www.edgeprop.sg/">EdgeProp.sg</a></strong> add analytical tools:</p><ul><li><p>RealSmart lets you filter projects by tenure, MRT proximity, and price trends. ( I like their <a href="https://www.realsmart.sg/map">map view</a>)</p></li><li><p>EdgeProp visualises transaction charts, rental yields, and floor plans.</p></li></ul><p>For deeper context:</p><ul><li><p><strong><a href="https://eservice.ura.gov.sg/maps/?service=MP">URA SPACE</a></strong> reveals future zoning and land use.</p></li><li><p><strong><a href="https://www.onemap.gov.sg/#/SchoolQueryInfo">OneMap School Query</a></strong> checks 1 km school distances.</p></li><li><p><strong><a href="https://www.openstreetmap.org/">OpenStreetMap</a></strong> shows small-scale details like paths and noise sources often missing from Google Maps.</p></li></ul><h3>Understanding Your Demand</h3><p>Write down your <strong>must-haves</strong> and <strong>good-to-haves</strong> &#8212; proximity to work, schools, transport, or food; age of property; unblocked view; tenure; and affordability. Once you define them, you&#8217;ll realise only a handful of properties truly qualify. That&#8217;s a feature, not a problem.</p><p>Be patient: the property market is illiquid. Treat the process as discovery &#8212; each viewing teaches you more about your real preferences and local pricing.</p><h3>Valuation Approaches</h3><p>There&#8217;s no single right way to value a property, but triangulating from multiple methods gives confidence.</p><h4>1. Comparative Method</h4><p>Use recent transactions of comparable units. Control for tenure, remaining lease, floor level, and transaction recency.</p><p><br>If data is sparse, look at neighbouring projects, adjusting for age, distance to MRT, and facilities. For enthusiasts, a <strong>hedonic pricing model</strong> &#8212; regression analysis on past transactions &#8212; can quantify these adjustments. For most buyers, eyeballing a dozen recent transactions within the same development is practical and reliable.</p><h4>2. Seller-Cost Perspective</h4><p>Estimate what return the seller might expect. Take the seller&#8217;s purchase price and compound it by the <strong><a href="https://eservice.ura.gov.sg/property-market-information/pmiResidentialTimeseries">URA Private Residential Property Price Index</a></strong> to approximate a fair &#8220;cost-plus&#8221; value. If the asking price far exceeds that benchmark without clear upgrades or market shifts, negotiate.</p><h4>3. Discounted Cash-Flow Perspective</h4><p>Value the property as an income-producing asset: calculate the present value of future real rental yields (adjusted for inflation) over the remaining lease, discounted by your required rate of return &#8212; roughly the long-term property-index growth minus inflation. This lens helps investors compare property returns to other asset classes.</p><p>By comparing these three perspectives, you get a realistic <strong>fair-value range</strong>, rather than relying solely on agents&#8217; optimism.</p><div><hr></div><h2>4. Making a Purchase</h2><p>Once you&#8217;ve found your property and agreed on terms, the process turns procedural &#8212; a series of payments and signatures. Here&#8217;s the typical flow.</p><h3>Step 1: Property Inspection</h3><p>Hire a professional inspector before committing. The report forms your basis for <strong>negotiating repairs or price adjustments</strong>, which must be resolved <em>before</em> the OTP is signed.</p><h3>Step 2: Secure Bank Support (Part 1 &#8211; IPA)</h3><p>Apply for <strong>In-Principle Approvals (IPA)</strong> with several banks. This pre-qualifies your loan amount and gives leverage when comparing rates.</p><h3>Step 3: Negotiate Terms with Seller</h3><p>Finalise every key term <strong>before signing the Option to Purchase (OTP)</strong> &#8212; price, repair obligations, completion date (usually 8&#8211;12 weeks post-exercise), option period (14&#8211;21 days), and also the handover date. Once signed, these are binding.</p><h3>Step 4: Sign the Option to Purchase (OTP)</h3><p>Pay a <strong>1 % option fee</strong> to reserve the unit. You now have the right &#8212; but not yet the obligation &#8212; to buy within the option window.</p><h3>Step 5: Engage a Lawyer</h3><p>Engage a <strong>conveyancing lawyer</strong> immediately after signing. They&#8217;ll conduct title checks, lodge a caveat, and coordinate with both bank and seller.</p><h3>Step 6: Secure Bank Support (Part 2 &#8211; Loan Application)</h3><p>Submit your <strong>formal loan application</strong>. After valuation, the bank issues a <strong>Letter of Offer</strong> detailing rate type, lock-in, and prepayment clauses. Review carefully.</p><h3>Step 7: Exercise the OTP</h3><p>Once financing is confirmed, <strong>exercise the OTP</strong> by paying the remaining <strong>4 % deposit</strong> and signing the agreement. This legally binds the purchase.</p><h3>Step 8: Pay Stamp Duties</h3><p>Within <strong>14 days</strong> of exercising, pay <strong>BSD</strong> and, if applicable, <strong>ABSD</strong>. Your lawyer typically manages the IRAS submission.</p><h3>Step 9: Conduct Final Walkthrough (Pre-Completion)</h3><p>Before completion, do a <strong>final inspection</strong> to ensure all agreed repairs are completed and the unit&#8217;s condition matches what was promised.</p><h3>Step 10: Completion and Final Payment</h3><p>Roughly <strong>8&#8211;12 weeks</strong> after exercise, the transaction completes. You&#8217;ll pay the <strong>remaining 20 % downpayment</strong>, and the bank disburses the loan to the seller. Ownership transfers when your lawyer receives the keys.</p><h3>Step 11: Post-Completion Tasks</h3><p>Set up <strong>MCST billing</strong>, arrange <strong>home insurance</strong>, and confirm <strong>property-tax</strong> payment schedules.</p><p>The full process typically spans three to four months from offer to key collection. Keeping a detailed spreadsheet of milestones, payment amounts, and due dates makes the experience smooth and predictable.</p><div><hr></div><h2>5. Other Topics Worth Your Attention</h2><h3>Layout and Efficiency</h3><p>Two units with identical floor areas can feel entirely different. Prioritise <strong>usable layout</strong> over raw square footage. Efficient layouts minimise corridors, have proper separation between bedrooms and living areas, and ensure natural cross-ventilation. When viewing, bring a floor plan and imagine furniture placement; awkward corners often reveal inefficient design.</p><h3>Harmonisation of Gross Floor Area (GFA)</h3><p>Starting <strong>September 2024</strong>, URA introduced <strong>harmonised definitions of floor area</strong> across developments. Under this rule, certain spaces like <strong>air-conditioner ledges, planter boxes, and other non-liveable areas are no longer counted</strong> in the strata area used for sale.</p><p>This means new projects launched under the harmonised guidelines will show <strong>smaller strata areas</strong> compared to older ones, even if the internal usable space is similar. Buyers should therefore compare <strong>layout efficiency and liveable area</strong>, not just the headline PSF. A newer 900 sq ft unit under harmonisation may feel similar in size to a 950 sq ft pre-harmonisation unit.</p><h3>Mortgage Rates</h3><p>When taking a mortgage, decide between <strong>fixed</strong> and <strong>floating</strong> rates. Fixed provides stability during the lock-in period; floating follows SORA and moves with market conditions. Your choice depends on your risk tolerance and cash-flow flexibility.</p><h3>Home Insurance and Maintenance</h3><p>After completion, secure both <strong>fire insurance</strong> (mandatory for mortgaged properties) and <strong>home-contents insurance</strong>. Also budget for recurring <strong>MCST fees</strong> and an emergency fund for unexpected repairs or rate hikes.</p><h3>Renovation and Timing</h3><p>Plan renovation early. Obtain several quotes and prioritise functional work before cosmetic upgrades. Align renovation timelines with key collection to avoid paying rent and mortgage simultaneously.</p><div><hr></div><h2>6. Tricks and Tips</h2><p>A few practical lessons made a real difference during my own purchase.</p><p><strong>1. Clarify who you&#8217;re dealing with.</strong><br>Sometimes the &#8220;agent&#8221; is actually the <strong>seller</strong> posing as their own representative. Always open with: <em>&#8220;Are you the agent or the seller?&#8221;</em> Transparency changes the tone immediately.</p><p><strong>2. Understand the commission structure.</strong><br>Without a buyer&#8217;s agent, there&#8217;s room to negotiate because the seller&#8217;s side saves that 1 % commission. The seller&#8217;s agent can&#8217;t rebate you directly, but the <strong>price can often be adjusted</strong> if the seller agrees to share the savings. Think of the deal as a <strong>multi-party negotiation</strong> &#8212; not just buyer versus seller.</p><p><strong>3. Negotiate patiently.</strong><br>Seller agents are skilled at extracting your &#8220;best offer&#8221; without reducing theirs. Increase bids slowly, and let silence work for you. Time is leverage.</p><p><strong>4. Negotiate beyond price.</strong><br>Completion date, extension of stay, inclusion of furniture, or electronics can all be part of the negotiation.</p><p><strong>5. Study sunlight.</strong><br>When viewing, you only see a snapshot of the year. Check the <strong><a href="https://www.suncalc.org/#/1.3534,103.8165,11/2025.10.14/21:05/1/0">sun&#8217;s path</a></strong> to understand how light and heat will shift through the seasons.</p><p><strong>6. Visit multiple times.</strong><br>View the property at different hours &#8212; before work, after work, and near school start if close to a school. Observe noise, traffic, and lighting. Visualise daily life before committing.</p><p><strong>7. Compare mortgage rates</strong></p><p>The initial quote is rarely the lowest rate. Obtain several offers from different banks, then negotiate with alternative banks to encourage your primary bank to match the best offer. In addition to the mortgage rate, consider negotiating other terms such as the lock-in period, free conversion, and additional perks.</p><div><hr></div><h2>7. Reflections</h2><p>Looking back, buying a resale condo in Singapore taught me the value of patience, preparation, and perspective.</p><p>The process rewards those who do their homework: understanding financing rules, using data to spot fair value, and planning cash flow down to the week. But it also demands intuition &#8212; knowing when a property &#8220;feels right,&#8221; and when to walk away.</p><p>A property purchase isn&#8217;t only a financial decision; it&#8217;s a lifestyle and time-management decision wrapped in one. Balancing analysis with lived imagination &#8212; picturing how you&#8217;ll actually inhabit the space &#8212; turns a stressful transaction into a meaningful one.</p><p>If there&#8217;s one lesson I&#8217;d emphasise, it&#8217;s this: <strong>clarity reduces anxiety.</strong> Know your numbers, understand the timeline, and negotiate at your own pace. With that, the process becomes less of a gamble and more of a disciplined journey toward a home that truly fits.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Viewing Loan Core as a State Machine]]></title><description><![CDATA[How computational theory reveals the hidden order behind lending systems.]]></description><link>https://www.zhizhi-gewu.com/p/viewing-loan-core-as-a-state-machine</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/viewing-loan-core-as-a-state-machine</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 19 Oct 2025 03:11:18 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!k0OG!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Loan management systems &#8212; or <em>loan core</em> &#8212; form the backbone of financial institutions. They govern the business logic behind loan creation, approval, disbursement, and repayment &#8212; in short, the entire loan cycle.</p><p>In abstract form, a loan core defines the <em>states</em> of a loan, the <em>actions</em> that can be taken (by users or by the system through auto-triggers), and the <em>transitions</em> between these states. When I began learning more about computational theory, it struck me that a loan core is, at its essence, a <strong>state machine</strong>.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>It has to be &#8212; because a computer itself is a state machine. Viewing loan core from the lens of computational theory not only gives a formal framework for understanding how it works, but also helps navigate the various implementations across institutions. Beneath all the different designs and data structures, the logic can be abstracted into one thing: a finite state machine.</p><div><hr></div><h3><strong>A Simplified Loan Core Example</strong></h3><p>Consider a minimal version of a loan core system.</p><p>A new loan starts in an initial state and moves through a sequence of states based on user actions or automated events:</p><pre><code><code>[start] &#8594; (create) &#8594; [Pending] &#8594; (approve) &#8594; [Approved] &#8594; (disburse) &#8594; [Active] &#8594; (full repayment) &#8594; [Closed]
                            &#8600; (reject)
                              [Rejected]
</code></code></pre><p>Each arrow represents a <em>transition</em> triggered by an event such as &#8220;approve&#8221; or &#8220;disburse.&#8221; The machine ensures that only valid transitions occur &#8212; for example, a loan can&#8217;t be disbursed before it&#8217;s approved.</p><p>A <strong>finite state machine (FSM)</strong> is, by definition, a machine that &#8212; at any given moment &#8212; is in a single state. When it receives an input, it looks up its transition table to determine the next state, and possibly what output or action to produce.</p><div><hr></div><h3><strong>Formal Definition</strong></h3><p>Formally, a finite state machine can be defined as a five-tuple:</p><div class="latex-rendered" data-attrs="{&quot;persistentExpression&quot;:&quot;M = (Q, \\Sigma, \\delta, q_0, F)&quot;,&quot;id&quot;:&quot;XLIDIUOHKN&quot;}" data-component-name="LatexBlockToDOM"></div><p></p><p>where:</p><ul><li><p><strong>Q</strong> is the finite set of states.</p></li><li><p><strong>&#931;</strong> is the finite set of possible inputs (the <em>alphabet</em>).</p></li><li><p><strong>&#948;</strong> is the transition function: ( Q &#215; &#931; &#8594; Q ).</p></li><li><p><strong>q&#8320;</strong> &#8712; <strong>Q</strong> is the start state.</p></li><li><p><strong>F</strong> &#8838; <strong>Q</strong> is the set of accept (or terminal) states.</p></li></ul><p>Applying this to our loan core system:</p><ul><li><p><strong>Q = { null, pending, approved, rejected, active, closed }</strong></p></li><li><p><strong>&#931; = { create, approve, reject, disburse, full_repayment }</strong></p></li><li><p><strong>q&#8320; = null (start)</strong></p></li><li><p><strong>F = { rejected, closed }</strong></p></li></ul><div><hr></div><h3><strong>Transition Table</strong></h3><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Polq!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Polq!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png 424w, https://substackcdn.com/image/fetch/$s_!Polq!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png 848w, https://substackcdn.com/image/fetch/$s_!Polq!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png 1272w, https://substackcdn.com/image/fetch/$s_!Polq!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Polq!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png" width="1456" height="570" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:570,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:42424,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.zhizhi-gewu.com/i/176534520?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Polq!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png 424w, https://substackcdn.com/image/fetch/$s_!Polq!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png 848w, https://substackcdn.com/image/fetch/$s_!Polq!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png 1272w, https://substackcdn.com/image/fetch/$s_!Polq!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0759b136-cf3b-4018-abb8-497f98b8bd57_1552x608.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>This simple table captures the essence of loan lifecycle logic &#8212; an ordered system of possible transitions bounded by rules. In practice, real-world loan cores are <strong>extended finite state machines (EFSMs)</strong>, enriched with context variables, conditions, and side effects (for example, disbursing funds, updating ledgers, or sending notifications). But at their core, they remain recognizable as state machines &#8212; systems that transition deterministically between defined states.</p><div><hr></div><h2><strong>From Theory to Reality: Apache Fineract</strong></h2><p>One open-source example that demonstrates this beautifully is <strong>Apache Fineract</strong>, a microfinance and loan management platform.<br>Fineract implements an extended state machine for loan lifecycle management. Each state (like <em>Approved</em>, <em>Active</em>, or <em>Overpaid</em>) can respond to a wide range of <em>events</em> &#8212; such as disbursal, repayment, write-off, or reschedule.</p><p>The following <strong>state transition matrix</strong> illustrates how complex this system becomes once we consider real-world business rules.</p><h3><strong>State Transition Matrix</strong></h3><h4>Complete &#948;(Q, &#931;) Matrix</h4><p>This matrix shows <strong>all possible (state, event)</strong> combinations. Each cell contains the resulting state or an indicator of how the system handles that input. (This summary is genreated by Claude Code)</p><pre><code><code>Legend:
&#10003; = Valid transition (always allowed)
C = Conditional transition (depends on context)
- = Invalid/blocked transition
A = Auto-transition (system-triggered)

Event Abbreviations:
CR=CREATED, RJ=REJECTED, AP=APPROVED, WD=WITHDRAWN, DB=DISBURSED
AU=APPROVAL_UNDO, DU=DISBURSAL_UNDO, CP=CHARGE_PAYMENT, RW=REPAYMENT_OR_WAIVER
RF=REPAID_IN_FULL, WO=WRITE_OFF, WU=WRITE_OFF_UNDO, RS=RESCHEDULE
OV=OVERPAYMENT, CA=CHARGE_ADDED, CX=CHARGE_ADJUSTMENT, CB=CHARGEBACK
AT=ADJUST_TRANSACTION, IT=INITIATE_TRANSFER, RT=REJECT_TRANSFER
WT=WITHDRAW_TRANSFER, CR=CREDIT_BALANCE_REFUND
</code></code></pre><h4>Main Transition Matrix</h4><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!k0OG!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!k0OG!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png 424w, https://substackcdn.com/image/fetch/$s_!k0OG!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png 848w, https://substackcdn.com/image/fetch/$s_!k0OG!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png 1272w, https://substackcdn.com/image/fetch/$s_!k0OG!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!k0OG!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png" width="1456" height="445" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:445,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:87838,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.zhizhi-gewu.com/i/176534520?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!k0OG!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png 424w, https://substackcdn.com/image/fetch/$s_!k0OG!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png 848w, https://substackcdn.com/image/fetch/$s_!k0OG!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png 1272w, https://substackcdn.com/image/fetch/$s_!k0OG!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F88d19fc2-f07e-4084-bee1-8cc9c8004658_1970x602.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><pre><code><code>From: SUBMITTED_AND_PENDING_APPROVAL (100)
&#9500;&#9472; LOAN_REJECTED         &#8594; REJECTED (500)
&#9500;&#9472; LOAN_APPROVED         &#8594; APPROVED (200)
&#9492;&#9472; LOAN_WITHDRAWN        &#8594; WITHDRAWN_BY_CLIENT (400)

From: APPROVED (200)
&#9500;&#9472; LOAN_DISBURSED        &#8594; ACTIVE (300)
&#9492;&#9472; LOAN_APPROVAL_UNDO    &#8594; SUBMITTED_AND_PENDING_APPROVAL (100)

From: ACTIVE (300)
&#9500;&#9472; LOAN_DISBURSAL_UNDO   &#8594; APPROVED (200)
&#9500;&#9472; WRITE_OFF_OUTSTANDING &#8594; CLOSED_WRITTEN_OFF (601)
&#9500;&#9472; LOAN_RESCHEDULE       &#8594; CLOSED_RESCHEDULE_OUTSTANDING_AMOUNT (602)
&#9500;&#9472; LOAN_INITIATE_TRANSFER&#8594; TRANSFER_IN_PROGRESS (303)
&#9500;&#9472; REPAID_IN_FULL (auto) &#8594; CLOSED_OBLIGATIONS_MET (600) [if balance=0, charges paid]
&#9492;&#9472; LOAN_OVERPAYMENT (auto)&#8594; OVERPAID (700) [if totalOverpaid&gt;0]

From: OVERPAID (700)
&#9500;&#9472; LOAN_CREDIT_BALANCE_REFUND (auto) &#8594; CLOSED_OBLIGATIONS_MET (600) [if refunded]
&#9500;&#9472; LOAN_CHARGE_PAYMENT   &#8594; ACTIVE (300) [if creates outstanding]
&#9500;&#9472; LOAN_REPAYMENT_OR_WAIVER &#8594; ACTIVE (300) [if creates outstanding]
&#9500;&#9472; LOAN_CHARGEBACK       &#8594; ACTIVE (300)
&#9492;&#9472; REPAID_IN_FULL        &#8594; CLOSED_OBLIGATIONS_MET (600) [if conditions met]

From: CLOSED_OBLIGATIONS_MET (600)
&#9500;&#9472; LOAN_DISBURSED        &#8594; ACTIVE (300) [multi-disburse only]
&#9500;&#9472; LOAN_CHARGE_ADDED     &#8594; ACTIVE (300)
&#9500;&#9472; LOAN_CHARGEBACK       &#8594; ACTIVE (300)
&#9500;&#9472; LOAN_CHARGE_ADJUSTMENT&#8594; OVERPAID (700) [if creates overpayment]
&#9500;&#9472; LOAN_CHARGE_PAYMENT   &#8594; ACTIVE (300) [if creates outstanding]
&#9500;&#9472; LOAN_REPAYMENT_OR_WAIVER &#8594; ACTIVE (300) [if creates outstanding]
&#9500;&#9472; LOAN_ADJUST_TRANSACTION &#8594; ACTIVE (300) or OVERPAID (700) [depends on result]
&#9492;&#9472; LOAN_OVERPAYMENT (auto)&#8594; OVERPAID (700) [if totalOverpaid&gt;0]

From: CLOSED_WRITTEN_OFF (601)
&#9500;&#9472; WRITE_OFF_OUTSTANDING_UNDO &#8594; ACTIVE (300)
&#9492;&#9472; LOAN_ADJUST_TRANSACTION &#8594; ACTIVE (300) or OVERPAID (700) [depends on result]

From: CLOSED_RESCHEDULE_OUTSTANDING_AMOUNT (602)
&#9492;&#9472; LOAN_ADJUST_TRANSACTION &#8594; ACTIVE (300) or OVERPAID (700) [depends on result]

From: TRANSFER_IN_PROGRESS (303)
&#9500;&#9472; LOAN_REJECT_TRANSFER  &#8594; TRANSFER_ON_HOLD (304)
&#9492;&#9472; LOAN_WITHDRAW_TRANSFER&#8594; ACTIVE (300)

From: REJECTED (500)
&#9492;&#9472; [No transitions - terminal]

From: WITHDRAWN_BY_CLIENT (400)
&#9492;&#9472; [No transitions - terminal]

From: TRANSFER_ON_HOLD (304)
&#9492;&#9472; [No transitions - terminal]</code></code></pre><div><hr></div><h3><strong>Seeing Systems as State Machines</strong></h3><p>Modeling a system as a state machine is not just an academic exercise &#8212; it&#8217;s a way of <em>thinking</em>. It forces every assumption about what can happen, when, and under what conditions to become explicit. The loan core stops being a black box of code and business rules, and turns into a living map of states and transitions. You can reason about it, test it, simulate it, and explain it. Most importantly, it gives teams a shared grammar for complexity &#8212; engineers, risk analysts, and operations staff can finally point to the same table and agree on what &#8220;Active&#8221; or &#8220;Closed&#8221; really means.</p><div><hr></div><h3><strong>Beyond the State Machine</strong></h3><p>The state machine captures the <em>skeleton</em> of the loan core &#8212; the structure of states and valid transitions. But the living body of the system includes many other organs working in sync.</p><p><br>At the heart of it are <strong>transaction functions</strong> that handle money movement: <em>disbursement</em>, <em>repayment</em>, <em>fees</em>, and <em>penalties</em>. Each of these operations changes not only account balances but also the loan&#8217;s internal context &#8212; triggering new transitions in the state machine.</p><p>Surrounding this are <strong>repayment schedule generators</strong>, which determine expected installments and interest accruals; and <strong>fee and penalty managers</strong>. Beyond the core logic, other crucial layers complete the ecosystem:</p><ul><li><p><strong>Account management</strong>, which maintains borrower and product relationships.</p></li><li><p><strong>Accounting</strong>, which records journal entries and enforces financial integrity.</p></li><li><p><strong>API and service layers</strong>, which expose the system&#8217;s functions to external modules like mobile apps or credit platforms.</p></li></ul><p>Thinking in FSM terms doesn&#8217;t replace these subsystems &#8212; it organizes them. The state machine becomes the <em>control plane</em> that orchestrates when and how these modules act. By defining the legal transitions clearly, every financial operation &#8212; from posting a repayment to writing off a loan &#8212; has a well-defined place in the broader choreography.</p><div><hr></div><h3><strong>Further Reading</strong></h3><p>Much of the formalism behind state machines and computation comes from the field of <strong>automata theory</strong> &#8212; the mathematical study of what can be computed and how.</p><p><br>For readers who want to go deeper into this foundation, <em>Michael Sipser&#8217;s</em> classic textbook, <em>Introduction to the Theory of Computation (Third Edition)</em>, remains one of the clearest and most intuitive introductions. It covers finite automata, regular languages, Turing machines, and the limits of computation &#8212; ideas that quietly shape how systems like loan cores can be reasoned about, formalized, and proven correct.</p><p></p><p>For those keen to bridge theory with real-world patterns, I highly recommend the chapter <strong>&#8220;State&#8221;</strong> on the site <em>Game Programming Patterns</em> by Robert Nystrom (available online at <a href="https://gameprogrammingpatterns.com/state.html?utm_source=chatgpt.com">gameprogrammingpatterns.com/state.html</a>). It introduces <strong>finite state machines (FSMs)</strong> and the related <strong>State design pattern</strong> in the context of game development&#8212;illustrating how the abstract mathematical concepts map directly into software architecture and everyday programming.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Can Publication Data Predict Technology Breakthroughs? Analyzing 500+ Years of Innovation]]></title><description><![CDATA[How Ideas Appear in Print Before They Transform the World]]></description><link>https://www.zhizhi-gewu.com/p/can-publication-data-predict-technology</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/can-publication-data-predict-technology</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sun, 12 Oct 2025 03:36:22 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!YYBK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>What if we could see the future of technology by looking at what people wrote about in the past? This isn&#8217;t science fiction&#8212;it&#8217;s data science applied to historical publishing patterns.</p><p>I built <a href="https://github.com/kychanbp/google_ngram_gpts">an interactive tool</a> to test a fascinating hypothesis: <strong>publication frequency increases before major technology takeoff</strong>. By analyzing 500+ years of text from Google Books, we can watch ideas bubble up in the public consciousness <em>before</em> they transform the world.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h2><strong>The Hypothesis</strong></h2><p>The core idea is simple but powerful: when a transformative technology is about to &#8220;take off,&#8221; people write about it more. Scientists publish papers, engineers write manuals, journalists cover developments, and the public discusses implications&#8212;all leaving traces in the published record.</p><p>But does this pattern hold across different eras? Can we see the same signal before the printing press (1455), the steam engine (1760), and artificial intelligence (2012)?</p><p>That&#8217;s what this project sets out to explore.</p><h2><strong>The Technologies: 9 General Purpose Technologies</strong></h2><p>I selected nine transformative technologies that fundamentally reshaped human civilization. These aren&#8217;t just inventions&#8212;they&#8217;re <strong>General Purpose Technologies (GPTs)</strong>: innovations so fundamental that they enable countless downstream applications.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!oU0C!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!oU0C!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png 424w, https://substackcdn.com/image/fetch/$s_!oU0C!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png 848w, https://substackcdn.com/image/fetch/$s_!oU0C!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!oU0C!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!oU0C!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png" width="1456" height="999" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/ca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:999,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:269869,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.zhizhi-gewu.com/i/175315712?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!oU0C!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png 424w, https://substackcdn.com/image/fetch/$s_!oU0C!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png 848w, https://substackcdn.com/image/fetch/$s_!oU0C!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png 1272w, https://substackcdn.com/image/fetch/$s_!oU0C!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fca203ab9-0683-4de0-93fe-75ce03d8e1a7_1574x1080.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p></p><p>*Note: AI&#8217;s 2012 date marks the technical breakthrough (deep learning revolution), though economic diffusion across industries is still in early stages.</p><h2><strong>The Tool: Interactive Historical Analysis</strong></h2><p>I built a pure frontend web application that makes this analysis accessible to anyone. No installation, no backend&#8212;just open it in your browser and explore 500 years of innovation.</p><p><strong>Key Features:</strong></p><ul><li><p><strong>Customizable search terms</strong>: Each technology tracks multiple related terms (e.g., &#8220;artificial intelligence,&#8221; &#8220;machine learning,&#8221; &#8220;deep learning&#8221;). You can add or remove terms to refine the analysis</p></li><li><p><strong>Flexible time range</strong>: Use sliders to zoom into specific historical periods (1500-2020)</p></li><li><p><strong>Takeoff markers</strong>: Dashed vertical lines show when each technology historically &#8220;took off&#8221;</p></li><li><p><strong>Data export</strong>: Download the underlying data as CSV for your own analysis</p></li></ul><p>The tool tracks how frequently each technology&#8217;s related terms appear in published books over time, using data from Google&#8217;s Ngram Viewer (which has digitized millions of books).</p><h2><strong>Methodology: From Books to Insights</strong></h2><h3><strong>Data Source: Google Ngrams</strong></h3><p>Google Books has digitized over 40 million books. The Ngram Viewer tracks word and phrase frequency across this massive corpus, providing a quantitative lens on cultural and intellectual history.</p><p>For each technology, I query multiple related search terms. For example, &#8220;Computers &amp; Semiconductors&#8221; tracks: computer, semiconductor, transistor, integrated circuit, microprocessor, and software.</p><h3><strong>Aggregation Strategy: Maximum Frequency</strong></h3><p>Here&#8217;s a critical methodological choice: how do we combine multiple search terms into a single trend line?</p><ul><li><p><strong>Average</strong>: Would dilute the signal by averaging high-frequency terms with low-frequency ones, underrepresenting the peak of public discourse</p></li><li><p><strong>Sum</strong>: Would artificially inflate technologies with more search terms (10 terms at 0.01% each sums to 0.10%, while 5 terms at 0.01% sums to 0.05%)</p></li><li><p><strong>Maximum</strong>: Takes the highest frequency in each year&#8212;<strong>this is what I chose</strong></p></li></ul><p>Using MAX ensures fair comparison across technologies. It captures the &#8220;peak signal&#8221; of public discourse in each year&#8212;the most prominent way people talked about each innovation&#8212;regardless of vocabulary breadth. Some technologies have many synonyms (AI: machine learning, deep learning, neural networks), others have few (railways: railroad, train). MAX measures cultural prominence, not vocabulary diversity.</p><h3><strong>Data Quality Filtering</strong></h3><p>Google&#8217;s OCR isn&#8217;t perfect, especially for older texts. To remove noise, I filter out data points with fewer than 1,000 absolute matches. This eliminates spurious spikes from misread characters while preserving genuine historical trends.</p><h2><strong>What The Data Shows</strong></h2><p>The results reveal striking patterns about how technologies move from ideas to widespread adoption&#8212;and eventually fade from public discourse.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!YYBK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!YYBK!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png 424w, https://substackcdn.com/image/fetch/$s_!YYBK!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png 848w, https://substackcdn.com/image/fetch/$s_!YYBK!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png 1272w, https://substackcdn.com/image/fetch/$s_!YYBK!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!YYBK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png" width="1456" height="971" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:971,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:305499,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.zhizhi-gewu.com/i/175315712?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!YYBK!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png 424w, https://substackcdn.com/image/fetch/$s_!YYBK!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png 848w, https://substackcdn.com/image/fetch/$s_!YYBK!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png 1272w, https://substackcdn.com/image/fetch/$s_!YYBK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66420f2b-6d1c-4c3e-ac7c-7bfa4fe220de_2400x1600.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h3><strong>Pattern 1: The Rise-Peak-Decline Lifecycle</strong></h3><p>Every mature technology follows a consistent arc: publication frequency rises during innovation and early adoption, peaks, then <strong>declines as the technology becomes normalized</strong>.</p><ul><li><p><strong>Internal Combustion &amp; Aviation</strong>: Publications surge from 1900&#8211;1930, then gradually decline as cars become mundane</p></li><li><p><strong>Computers &amp; Semiconductors</strong>: Peak interest around 1985&#8211;1990, then sharp decline through the 2000s</p></li><li><p><strong>Networks &amp; Internet</strong>: Explosive rise through the 1990s, peak around 2000&#8211;2005, then rapid decline</p></li></ul><p>This pattern suggests publications track <em>novelty and transformation</em>, not ubiquity. Once a technology is embedded in daily life, people stop writing books about it.</p><h3><strong>Pattern 2: Publications Surge Before Takeoff&#8212;The Hypothesis Holds</strong></h3><p>Across all technologies, <strong>publication frequency increases before the takeoff date</strong>, validating the core hypothesis:</p><ul><li><p><strong>Internal Combustion (1920 takeoff)</strong>: Publications begin rising around 1900-1910, well before mass adoption in the 1920s</p></li><li><p><strong>Computers (1965 takeoff)</strong>: Publications start their steep climb around 1960, anticipating the business computing boom</p></li><li><p><strong>Internet (1995 takeoff)</strong>: Publications surge from the early 1990s, ahead of mainstream Web adoption</p></li><li><p><strong>AI (2012 takeoff)</strong>: Publications rise from ~2000 onward, a decade before the deep learning breakthrough</p></li></ul><p>This consistent pattern suggests that <strong>ideas percolate through published discourse before they transform the economy</strong>. The intellectual groundwork&#8212;research papers, technical manuals, popular science books&#8212;precedes widespread commercial deployment.</p><p>The publication surge acts as a leading indicator: by the time a technology &#8220;takes off&#8221; economically, years of written discourse have already laid the conceptual foundation.</p><h3><strong>Pattern 3: Technology Succession is Visible</strong></h3><p>As one GPT&#8217;s publication frequency declines, another rises. The chart shows clear &#8220;handoffs&#8221;:</p><ul><li><p>Internal Combustion declines from the 1960s as Computers rise</p></li><li><p>Computers decline from the 1990s as Internet rises</p></li><li><p>Internet declines from the 2000s as AI rises</p></li></ul><p>Each technology doesn&#8217;t disappear&#8212;it becomes <em>background infrastructure</em> while attention shifts to the next transformative innovation.</p><h3><strong>Pattern 4: Technology Cycles Are Accelerating</strong></h3><p>Each successive technology reaches its publication peak <strong>faster</strong> than the previous one:</p><ul><li><p><strong>Internal Combustion</strong>: Gradual 30-year rise (1900&#8594;1930 peak)</p></li><li><p><strong>Computers</strong>: Steeper 25-year rise (1960&#8594;1985 peak)</p></li><li><p><strong>Internet</strong>: Explosive 10-year rise (1990&#8594;2000 peak)</p></li><li><p><strong>AI</strong>: Very steep rise, no peak yet (2000&#8594;2020+)</p></li></ul><p>This acceleration mirrors the broader pace of technological change. The Internet compressed decades of computer-era evolution into a single decade. The slope of each curve gets steeper, reflecting faster diffusion, faster adoption, and faster saturation of public discourse.</p><p><strong>Why cycles accelerate:</strong></p><ul><li><p>Information spreads faster (from print &#8594; journals &#8594; internet &#8594; social media)</p></li><li><p>Technologies build on existing infrastructure (Internet rode on computers and telecom)</p></li><li><p>Global markets enable simultaneous worldwide adoption</p></li><li><p>Venture capital and R&amp;D investment compress development timelines</p></li></ul><h3><strong>Pattern 5: AI is Unique&#8212;It&#8217;s Still Rising</strong></h3><p>Unlike every other technology, <strong>Artificial Intelligence shows no sign of peaking yet</strong>. Publications have risen continuously from ~2000 to 2020 and show no decline.</p><p>This suggests:</p><ul><li><p>AI is still in its transformative phase (not yet &#8220;normalized&#8221;)</p></li><li><p>The full scope of applications is still being explored</p></li><li><p>We may be living through the most intense period of AI discourse and development</p></li></ul><p>The contrast is stark: while Internet publications peaked 20 years ago and have since halved, AI publications continue their exponential climb.</p><h2><strong>Limitations &amp; Caveats</strong></h2><p>While this analysis reveals fascinating patterns, it&#8217;s important to understand what we&#8217;re <em>not</em> seeing:</p><h3><strong>The Book Publication Delay</strong></h3><p>This analysis only covers <strong>published books</strong>, which have a significant delay effect. From idea to published book typically takes:</p><ul><li><p>Academic books: 2-5 years (research &#8594; peer review &#8594; publication)</p></li><li><p>Technical manuals: 1-3 years (development &#8594; documentation &#8594; printing)</p></li><li><p>Trade books: 1-2 years (writing &#8594; editing &#8594; publishing)</p></li></ul><p>This means:</p><ul><li><p><strong>We&#8217;re seeing lagging indicators, not leading ones</strong>: By the time a technology appears frequently in books, it may have already &#8220;taken off&#8221; in labs, startups, or early adopter communities</p></li><li><p><strong>Journal articles come first</strong>: Academic papers appear years before books, so the &#8220;publication surge&#8221; likely started earlier than this data suggests</p></li><li><p><strong>Real-time discourse is invisible</strong>: Online discussions, preprints, blog posts, and social media conversations aren&#8217;t captured in Google Books data</p></li></ul><h3><strong>What This Means for Interpretation</strong></h3><p>The &#8220;publication frequency increases before takeoff&#8221; pattern might actually be:</p><ul><li><p>&#8220;Publication frequency increases <em>during early</em> takeoff&#8221; (books lag the actual innovation)</p></li><li><p>More accurately capturing when technologies become <em>mainstream topics</em> rather than when they&#8217;re <em>invented</em></p></li></ul><p>For truly predictive analysis, we&#8217;d need to supplement this with:</p><ul><li><p>Academic journal databases (Web of Science, arXiv)</p></li><li><p>Patent filings</p></li><li><p>Venture capital investment data</p></li><li><p>Online discourse metrics (if analyzing recent technologies)</p></li></ul><p><strong>Bottom line</strong>: This tool shows when technologies became part of the published conversation, which is valuable&#8212;but it&#8217;s not a crystal ball for predicting the <em>next</em> breakthrough.</p><h2><strong>Conclusion: Reading the Future in the Archives</strong></h2><p>This project reveals something profound: <strong>innovation leaves traces before it transforms the world</strong>. By analyzing what people wrote about, we can see the intellectual ferment that precedes technological revolution.</p><p>Of course, correlation isn&#8217;t causation. High publication frequency doesn&#8217;t <em>cause</em> technology takeoff&#8212;both are symptoms of growing interest, feasibility, and investment. But the pattern is clear enough to be useful.</p><p>For researchers, this methodology offers a quantitative complement to qualitative tech history. For investors and strategists, it suggests that publication trends might be a leading indicator of technological transformation. For the curious, it&#8217;s a fascinating way to explore how ideas become reality.</p><p><strong>What patterns do you see in the data?</strong> I&#8217;d love to hear your observations, critiques, and ideas for improving the analysis.</p><p>The future is being written right now&#8212;in papers, blog posts, and research repositories. With the right tools, we can learn to read those signals.</p><div><hr></div><p><em>Built with vanilla JavaScript, Plotly.js, and data from Google Ngrams. No build process, no dependencies, just pure frontend exploration. Open source and ready for your contributions.</em></p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[From Domestication to AI: What Each General Purpose Technology GPT Unlocked—and How to Use the Lessons]]></title><description><![CDATA[Lessons from past waves for the age of AI]]></description><link>https://www.zhizhi-gewu.com/p/from-domestication-to-ai-what-each</link><guid isPermaLink="false">https://www.zhizhi-gewu.com/p/from-domestication-to-ai-what-each</guid><dc:creator><![CDATA[KY John]]></dc:creator><pubDate>Sat, 04 Oct 2025 08:12:15 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/bd61b5c9-107a-4099-a09e-2a8b8c4d34fd_1024x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>To navigate the realm of Artificial Intelligence, examining history can be helpful. It is not the first instance of developing transformative technology in human history. According to Richard G. Lipsey, Kenneth I. Carlaw, and Clifford T. Bekar in <em>Economic Transformations: General Purpose Technologies and Long-Term Economic Growth</em>, a General Purpose Technology (GPT) is defined by its specific characteristics.</p><ul><li><p>Scope of improvements</p></li><li><p>Multiple uses</p></li><li><p>Massive spillover effect</p></li></ul><h2>Timeline of GPTs and what each wave unlocked</h2><ul><li><p><strong>10,000&#8211;4,000 BC &#8212; Domestication (plants &amp; animals).</strong> Surplus food enabled permanent settlements, urbanization, and occupational specialization. The plough, wheel, and animal traction amplified farm and transport scale. Close contact with animals created new disease regimes that reshaped population dynamics.</p></li></ul><ul><li><p><strong>c. 3200 BC &#8212; Writing and numeracy.</strong> Ledgers, contracts, and taxation became scalable because information could be stored and transmitted reliably. Writing turned information into infrastructure, expanding long-distance trade and state capacity.</p></li><li><p><strong>3300&#8211;1200 BC &#8212; Bronze metallurgy.</strong> Strong, standardized tools and weapons professionalized craft and warfare. Long-distance tin&#8211;copper trade formed early strategic supply chains and supported money use, codified law, and imperial expansion.</p></li><li><p><strong>1200 BC onward &#8212; Iron and steel.</strong> Cheaper, tougher iron tools diffused widely across farms and workshops while iron weapons broadened military capability. A long learning curve culminated in steel, transforming agriculture, construction, and the balance of state power.</p></li><li><p><strong>AD 800&#8211;1200 &#8212; Medieval agriculture bundle.</strong> The heavy plough, three-field rotation, and the horse-collar lifted yields and labor productivity on heavy soils. Village institutions coordinated teams and managed risk, fueling sustained population growth.</p></li><li><p><strong>AD 900&#8211;1300 &#8212; Water and wind mechanization.</strong> Mills converted inanimate power into industry-grade work in activities like fulling, sawing, and papermaking. Capital pooling and riparian law matured around these installations, while mechanization generated visible social frictions.</p></li><li><p><strong>1450s &#8212; Printing press.</strong> Movable type collapsed the cost of exact copying, accelerating literacy and the recombination of ideas. Scientific communication, standardized doctrine, and the Reformation spread on the back of cheap replication.</p></li><li><p><strong>1700s&#8211;early 1800s &#8212; Steam and the factory system.</strong> Inanimate power combined with precision tooling to enable continuous, paced production under one roof. Urban industry, shift work, and standardized parts reorganized labor and capital formation.</p></li><li><p><strong>1830s&#8211;1870s &#8212; Railways.</strong> High-speed, high-capacity transport integrated regional markets and slashed time&#8211;distance frictions. Time zones, telegraph dispatch, and large-scale bond finance emerged to coordinate and fund the network.</p></li><li><p><strong>Late 1800s&#8211;early 1900s &#8212; Electricity, chemicals, telegraph/telephone.</strong> Flexible, point-of-use electric power and laboratory science created versatile factories and new materials. Real-time communication synchronized services and management across distance.</p></li><li><p><strong>Early&#8211;mid 1900s &#8212; Internal combustion, oil, aviation; mass production and quality control.</strong> Cheap mobility and interchangeable parts enabled global supply chains and affordable consumer durables. Statistical quality control and disciplined workflows made scale efficient rather than sloppy.</p></li><li><p><strong>1950s&#8211;1970s &#8212; Computers and semiconductors.</strong> Programmable logic automated calculation and control while software ecosystems took shape. Data storage and processing reorganized offices and industry as compatibility standards spread.</p></li><li><p><strong>1980s&#8211;2000s &#8212; Networks and the internet &#8594; mobile/cloud.</strong> Universal protocols and browsers drove the near&#8211;zero marginal cost distribution of digital goods and services. Platform markets globalized services, shifting value toward identity, trust, and governance.</p></li><li><p><strong>2000s&#8211;present &#8212; Biotech/genomics; advanced materials; AI (candidate GPT).</strong> Design-rather-than-discover paradigms and data/compute-driven automation broaden what can be built across science and industry. The new bottlenecks center on energy availability, high-quality data, rigorous evaluation, and credible safety/standards.</p></li></ul><h2>The GPT Playbook</h2><ul><li><p><strong>GPTs are bundles, not widgets.</strong> Value arrives only when devices, skills, organizational design, and infrastructure click together. Allocate as much time and capital to these complements as to the core technology.</p></li><li><p><strong>Diffusion beats invention.</strong> Most gains come from many sectors retooling rather than from the first demo. Track adoption along supply chains and ecosystems, not just showcase pilots.</p></li><li><p><strong>Institutions co-evolve.</strong> Law, finance, and standards lower coordination costs as technology scales. Push standard-setting, liability clarity, and financing mechanisms early.</p></li><li><p><strong>Bottlenecks migrate.</strong> Each wave shifts the limiting factor&#8212;from power to transport to data and now to energy again. Re-scan constraints regularly and redirect investment to the new chokepoint.</p></li><li><p><strong>Learning curves win.</strong> Experience reliably drives costs down and quality up, favoring scalable &#8220;good-enough&#8221; solutions. Choose architectures and processes that iterate quickly and learn cheaply.</p></li><li><p><strong>Systems beat point solutions.</strong> Integrated packages like the factory system outperform isolated tools. Build end-to-end stacks and operating routines rather than feature islands.</p></li><li><p><strong>Adoption is uneven.</strong> Geography, vested interests, and path dependence stagger uptake across places and sectors. Time your bets for the S-curve middle and tailor rollouts to local constraints.</p></li><li><p><strong>Friction is normal.</strong> Skill churn and displacement accompany every productivity jump. Pair deployments with retraining, role redesign, and humane safety nets.</p></li><li><p><strong>Intangibles lead early.</strong> Software, data, and process changes appear before clean P&amp;L gains. Measure capability KPIs&#8212;cycle time, error rate, and throughput&#8212;alongside ROI.</p></li><li><p><strong>Expect the J-curve.</strong> Costs often rise before complements mature enough to pay off. Stage milestones and avoid canceling just before compounding starts.</p></li></ul><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.zhizhi-gewu.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading ZhiZhi Gewu! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item></channel></rss>