Cookbook

Network

Network Protocols and Associated Ports

HTML

Symbols:

(A) Arrows

(B) Greek Letters

(C) Fractions

(D) Roman Numerals

(E) Currency Symbols

(F) ★ ★

Designing a Table Layout in HTML and CSS

The following guide provides an overview of styling an HTML table using CSS for a balanced and aesthetically pleasing presentation. The table structure features subtle hover effects, clearly defined headers, and flexible column layouts, all adaptable to various content types. This refined approach ensures readability and organization within any web document.

CSS Style for Table

The CSS styling below applies essential formatting to create a simple yet polished table layout. This style includes adjustments for width, padding, border effects, and hover interactions, presenting content in a user-friendly, structured manner. Each CSS rule is carefully chosen to enhance visual clarity without overwhelming the content.

<style>
    .simple-table {
        width: 100%;
        border-collapse: collapse;
        margin: 20px 0;
        text-align: left;
    }
    .simple-table th, .simple-table td {
        padding: 3px 3px;
        border-bottom: 1px solid #ddd;
    }
    .simple-table th {
        background-color: #f2f2f2;
    }
    .simple-table tr:hover {
        background-color: #f5f5f5;
    }
    .simple-classification-header {
        background-color: #f8f8f8; /* Achromatic color for section headers */
        text-align: left;
        font-weight: bold;
    }
</style>

• .simple-table: Sets the table width to occupy 100% of the container, enables border collapse for seamless cell edges, and applies a margin for spacing.
• .simple-table th, .simple-table td: Defines padding within each cell and adds a light bottom border to visually separate rows.
• .simple-table th: Differentiates header cells with a subtle background color, helping to highlight column labels.
• .simple-table tr:hover: Adds a hover effect to rows, enhancing interactivity by changing the background color.
• .simple-classification-header: Provides a distinct style for classification headers, allowing further sectioning within the table as needed.

HTML Structure for Table

Below is the HTML table structure, where each column header and cell is generically labeled. This generic approach allows the table to adapt to a wide range of content without limiting it to specific data types.

<div style="overflow-x: auto;">
    <table class="simple-table">
        <thead>
            <tr>
                <th>First Column Header</th>
                <th>Second Column Header</th>
                <th>Third Column Header</th>
                <th>Fourth Column Header</th>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td>First Column Content</td>
                <td>Second Column Content</td>
                <td>Third Column Content</td>
                <td>Fourth Column Content</td>
            </tr>
            <tr>
                <td>First Column Content</td>
                <td>Second Column Content</td>
                <td>Third Column Content</td>
                <td>Fourth Column Content</td>
            </tr>
        </tbody>
    </table>
</div>

This structure includes a table wrapper with overflow-x: auto; to enable horizontal scrolling for responsive viewing on smaller screens.

Adding a Separator Row

Separator rows can be added by creating a table row with a single cell that spans multiple columns, using the colspan attribute. Adding a border style to this row, such as a thick top border, visually divides table sections.

Naive Separator

Separator rows can be added by creating a table row with a single cell that spans multiple columns, using the colspan attribute. Adding a border style to this row, such as a thick top border, visually divides table sections.

<tr>
    <td colspan="4" style="border-top: 2px solid #000;"></td>
</tr>

• colspan="4": Specifies that the cell spans four columns, effectively creating a single-cell row across the entire table width.
• style="border-top: 2px solid #000;": Applies a thick top border to the separator cell, making it stand out as a divider.

Simple Separator

Use the following code to add a simple separator row. This row spans across all columns and includes a thick top border to divide table sections.

<tr>
    <td colspan="4" style="border-top: 2px solid #000; background-color: #f0f0f0; padding: 4px; border-radius: 3px;">
    </td>
</tr>

• colspan="4": Specifies that the cell spans four columns, effectively creating a single-cell row across the entire table width.
• style="border-top: 2px solid #000;": Applies a thick top border to the separator cell.
• background-color: #fff8dc;: Adds a subtle background color to enhance visibility.
• padding: 4px;: Provides spacing within the separator cell.
• border-radius: 3px;: Rounds the corners of the separator cell for a smoother appearance.

Separator with Text

To add a separator row that includes descriptive text and a distinct background, use the following code. This enhances the visual separation by providing context or labeling for the upcoming section.

<tr>
    <td colspan="5" style="background-color: #f0f0f0; border-top: 2px solid #000; padding: 8px; text-align: center;">
        <strong> ~~~Contents~~~ </strong>
    </td>
</tr>

• colspan="5": Adjust the number of columns spanned to match your table's layout.
• background-color: #f0f0f0;: Provides a contrasting background to highlight the separator.
• border-top: 2px solid #000;: Adds a thick top border for clear separation.
• padding: 8px;: Increases the spacing within the separator cell for better readability.
• text-align: center;: Centers the text within the separator cell.
• <strong> ~~~Contents~~~ </strong>: Adds bold text to label the separator, making it stand out as a section heading.

Creating a Table with Merged Cells

To group table data, the rowspan and colspan attributes allow merging cells across rows or columns. For instance, grouping rows by classification or category enhances clarity and reduces redundancy.

<table class="simple-table">
  <tr>
      <th>Classification</th>
      <th>Category</th>
      <th>Port Number</th>
      <th>Protocol</th>
      <th>Description</th>
  </tr>

  <!-- Basic Classification Rows -->
  <tr>
      <td rowspan="7">Basic</td>
      <td rowspan="2"></td>
      <td>20, 21</td>
      <td>FTP (File Transfer Protocol)</td>
      <td>Transfers files between a client and server.</td>
  </tr>
  <tr>
      <td>22</td>
      <td>SSH (Secure Shell)</td>
      <td>Provides secure remote access.</td>
  </tr>
  <!-- More rows here -->
</table>

• rowspan="7": Expands the cell to span seven rows, grouping them under a single classification.
• colspan="2": Expands the cell to span two columns, used in cases like header sections or specific merged cells.

Highlighting Text with Inline Styles for Mild and Light Emphasis

• This paragraph has a mild blue highlight for emphasis.

  <span style="background-color: #f0f8ff; padding: 4px; border-radius: 3px;">
    This paragraph has a mild blue highlight for emphasis.
  </span>

• This paragraph has a mild green highlight for emphasis.

  <span style="background-color: #e6ffe6; padding: 4px; border-radius: 3px;">
    This paragraph has a mild green highlight for emphasis.
  </span>

• This paragraph has a mild yellow highlight for emphasis.

  <span style="background-color: #fff8dc; padding: 4px; border-radius: 3px;">
    This paragraph has a mild yellow highlight for emphasis.
  </span>

• This paragraph has a very light grey highlight, creating a subtle effect.

  <span style="background-color: #f9f9f9; padding: 4px; border-radius: 3px;">
    This paragraph has a very light grey highlight, creating a subtle effect.
  </span>

Working with <pre> Tags: No Scroll, Y-Scroll, X-Scroll

1. Non-Scrollable <pre> Block

   This version grows vertically with the content and allows text to wrap naturally. It does not restrict either width or height, so very long text may become unwieldy on the page.

   <pre style="white-space: pre-wrap;">
       Your content will wrap within the width of the block without creating a horizontal scroll bar. 
       However, if the content is too long vertically, the block will grow naturally, and no scroll will appear.
   </pre>

   • white-space: pre-wrap;
     • Preserves whitespace and line breaks like the default <pre>.
     • Allows text to wrap within the width of the container, preventing horizontal scrolling.
   • No max-height or overflow properties
     • The <pre> block expands naturally in the vertical direction.
     • Useful when you want the entire content to be visible without scroll bars.

2. Scrollable <pre> Block (Vertical Scrolling)

   Limiting the height and adding vertical scrolling can help maintain a compact layout. Content exceeding the defined max-height will be scrollable within the <pre> area.

   <pre style="white-space: pre-wrap; max-height: 200px; overflow-y: auto;">
   Your code or text goes here. The block will scroll vertically if it exceeds 200px in height.
   </pre>

   • white-space: pre-wrap;
     • Ensures long lines of text wrap to the next line, preventing a wide horizontal overflow.
   • max-height: 200px;
     • Limits the maximum height to 200px.
     • Creates a vertical scroll bar if content exceeds this height.
   • overflow-y: auto;
     • Adds a vertical scroll bar once content extends beyond the maximum height.

3. Scrollable <pre> Block (Horizontal Scrolling)

   Useful for code or text with very long lines. Setting overflow-x: auto; ensures a horizontal scroll bar appears rather than forcing the lines to wrap or stretching the page horizontally.

   <pre style="white-space: pre; overflow-x: auto;">
   Your long code or text goes here. If the line is too wide, a horizontal scroll bar will appear.
   This allows you to maintain the original formatting and indentation without wrapping.
   </pre>

   • white-space: pre;
     • Preserves all whitespace, tabs, and line breaks exactly as typed.
     • Keeps text on one line rather than wrapping.
   • overflow-x: auto;
     • Adds a horizontal scroll bar if text exceeds the width of the parent container.
     • Ensures the code or text formatting remains intact without forced wrapping.

Written on April 16, 2025

Invisible Link

In some cases, it may be useful to have a hyperlink that is visually hidden but still functional. This can be achieved using inline CSS to set the link’s color to match the background, remove the underline, and define the cursor behavior. The following code demonstrates how to make a link invisible.

<a href="./rough_drafts.html" style="color: rgba(211, 239, 224, 0); text-decoration: none; cursor: pointer;">&</a>

• color: rgba(211, 239, 224, 0);: Sets the link’s color to transparent using an rgba value with an alpha of 0, effectively making it invisible.
• text-decoration: none;: Removes the default underline typically applied to links, further ensuring that the link is invisible.
• cursor: pointer;: Retains the pointer cursor when hovering over the invisible link, making it functionally identifiable without being visible.

Dotted Horizontal Line

In some designs, a dotted horizontal line can be useful to subtly divide sections without a strong visual impact. This effect can be achieved by customizing the <hr> element's border properties using inline CSS. The following code demonstrates how to create a dotted horizontal line.

<hr style="border: 0; border-top: 1px dotted grey; margin-left: 10px; margin-right: 10px;">

• border: 0;: Removes the default border of the <hr> element, allowing for a custom style to be applied.
• border-top: 1px dotted grey;: Defines a single pixel thick, dotted line in grey color along the top edge of the element, creating the dotted horizontal line effect.
• margin-left: 10px; and margin-right: 10px;: Adds spacing on the left and right of the dotted line, adjusting its alignment within the page layout.

This code creates a subtle divider that can be used to delineate content sections while maintaining a minimalist appearance:

Aligning Text

(1) Using the align Attribute

The align attribute, though now deprecated in HTML5, is a simple way to align text within an HTML element. It can be set to values such as center, left, or right.

<p align="center">This paragraph is centered using the align attribute.</p>

• Easy to implement but not recommended for modern HTML as it does not separate content from presentation.
• Use CSS for better control and future compatibility.

(2) Using CSS text-align Property

The text-align CSS property provides a modern and flexible way to align text. It allows you to center, left-align, or right-align text with precision.

<p style="text-align: center;">This paragraph is centered using the text-align CSS property.</p>

• The text-align property is applied directly to the element using the style attribute or within a <style> block for reusability.
• Works consistently across all modern browsers.

CSS Styling Example for Reusability

You can define a reusable CSS class for centering text:

<style>
    .center-text {
        text-align: center;
    }
</style>
<p class="center-text">This paragraph uses a CSS class to center text.</p>

• Enhances code maintainability.
• Separates style from HTML structure, adhering to modern web development practices.

Deprecated Features in HTML5

In the evolution of web development, HTML5 has emerged as a standard that emphasizes clean, semantic markup and the separation of content, presentation, and behavior. The term "deprecated" in HTML5 refers to features—such as elements or attributes—that are no longer recommended for use. While these deprecated features may still function in some browsers for backward compatibility, they are marked for potential removal in future versions and are not guaranteed to work indefinitely. This guide explores what "deprecated" means in the context of HTML5, reasons behind deprecations, examples of deprecated features along with their recommended alternatives, and provides a summary table for quick reference.

• Separation of Content and Presentation: HTML5 promotes the separation of structure (HTML), styling (CSS), and behavior (JavaScript). This approach leads to cleaner code and easier maintenance.
• Enhanced Flexibility and Control: Modern CSS offers powerful tools for styling, including media queries, animations, and grid layouts, which are more flexible than outdated HTML attributes.
• Simplification and Efficiency: Removing outdated features reduces complexity within the HTML specification, encouraging developers to adopt more efficient methods that enhance performance and scalability.

<font face="Arial" size="4" color="blue">Sample Text</font>

<span style="font-family: Arial; font-size: 16px; color: blue;">Sample Text</span>

<center>This text is centered.</center>

<div style="text-align: center;">This text is centered.</div>

<img src="image.jpg" align="right">

<img src="image.jpg" style="float: right;">

<table bgcolor="lightblue">
    <tr>
        <td>Sample Text</td>
    </tr>
</table>

<table style="background-color: lightblue;">
    <tr>
        <td>Sample Text</td>
    </tr>
</table>

<table border="1">
    <tr>
        <td>Sample Text</td>
    </tr>
</table>

<table style="border: 1px solid black;">
    <tr>
        <td>Sample Text</td>
    </tr>
</table>

<applet code="SampleApplet.class" width="300" height="300"></applet>

<iframe src="path_to_content" width="300" height="300"></iframe>

<basefont face="Verdana" size="3" color="green">

<style>
    body {
        font-family: Verdana;
        font-size: 14px;
        color: green;
    }
</style>

<frameset cols="50%,50%">
    <frame src="frame1.html">
    <frame src="frame2.html">
</frameset>

<iframe src="frame1.html" style="width: 50%; height: 100%; float: left;"></iframe>
<iframe src="frame2.html" style="width: 50%; height: 100%; float: right;"></iframe>

<a name="bookmark">Bookmark</a>

<a id="bookmark">Bookmark</a>

<isindex prompt="Search:">

<form action="search.php" method="get">
    <label for="search">Search:</label>
    <input type="text" id="search" name="q">
    <button type="submit">Go</button>
</form>

Creating Illustrative Graphs with Chart.js

Creating interactive and visually appealing graphs on a webpage is streamlined through the utilization of HTML in conjunction with JavaScript libraries such as Chart.js. This guide elucidates the general procedures required to construct various types of charts, facilitating future reference for similar tasks.

Chart.js is a widely recognized open-source JavaScript library designed to facilitate the creation of aesthetically pleasing and interactive charts and graphs. Its versatility encompasses a multitude of chart types, coupled with extensive customization options to suit diverse visualization needs.

Setting Up Your HTML File

The foundational step involves establishing a basic HTML structure. The following template serves as a starting point:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Your Chart Title</title>
</head>
<body>
    <!-- Content will be added here -->
</body>
</html>

Including the Chart.js Library

Incorporating the Chart.js library is essential for rendering charts. This can be achieved by embedding a <script> tag that references the library via a Content Delivery Network (CDN):

<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>

This script tag should be placed either within the <head> section or immediately before the closing </body> tag.

Creating a Canvas Element

The <canvas> element serves as the rendering surface for the charts. Assigning a unique id to each canvas allows for precise targeting within JavaScript:

<canvas id="myChart" width="800" height="400"></canvas>

The width and height attributes can be adjusted to accommodate the desired chart dimensions.

Drawing Different Types of Charts

1. Line Chart

   Example: Plotting an exchange rate over time.

   HTML:

   <canvas id="lineChart" width="800" height="400"></canvas>

   JavaScript:

   <script>
   const ctx = document.getElementById('lineChart').getContext('2d');
   const lineChart = new Chart(ctx, {
       type: 'line',
       data: {
           labels: ['Date1', 'Date2', 'Date3', '...'],
           datasets: [{
               label: 'Dataset Label',
               data: [value1, value2, value3, ...],
               backgroundColor: 'rgba(54, 162, 235, 0.2)', // Fill color under the line
               borderColor: 'rgba(54, 162, 235, 1)', // Line color
               borderWidth: 2,
               fill: true, // Whether to fill under the line
               tension: 0.1 // Smoothness of the line
           }]
       },
       options: {
           responsive: true,
           plugins: {
               title: {
                   display: true,
                   text: 'Chart Title',
                   font: {
                       size: 18
                   }
               },
               legend: {
                   display: true,
                   position: 'top'
               }
           },
           scales: {
               x: {
                   title: {
                       display: true,
                       text: 'X-axis Label'
                   }
               },
               y: {
                   title: {
                       display: true,
                       text: 'Y-axis Label'
                   },
                   beginAtZero: false
               }
           }
       }
   });
   </script>

2. Bar Chart

   Example: Displaying estimated liquidity distribution.

   HTML:

   <canvas id="barChart" width="800" height="400"></canvas>

   JavaScript:

   <script>
   const ctx = document.getElementById('barChart').getContext('2d');
   const barChart = new Chart(ctx, {
       type: 'bar',
       data: {
           labels: ['Category1', 'Category2', '...'],
           datasets: [{
               label: 'Dataset Label',
               data: [value1, value2, ...],
               backgroundColor: ['color1', 'color2', ...]
           }]
       },
       options: {
           responsive: true,
           plugins: {
               title: {
                   display: true,
                   text: 'Chart Title'
               },
               legend: {
                   display: false
               }
           },
           scales: {
               y: {
                   beginAtZero: true,
                   title: {
                       display: true,
                       text: 'Y-axis Label'
                   }
               }
           }
       }
   });
   </script>

3. Pie Chart

   Example: Illustrating the proportion of a whole, such as a decrease in deposits.

   HTML:

   <canvas id="pieChart" width="400" height="400"></canvas>

   JavaScript:

   <script>
   const ctx = document.getElementById('pieChart').getContext('2d');
   const pieChart = new Chart(ctx, {
       type: 'pie',
       data: {
           labels: ['Segment1', 'Segment2', '...'],
           datasets: [{
               data: [value1, value2, ...],
               backgroundColor: ['color1', 'color2', ...],
               hoverOffset: 4
           }]
       },
       options: {
           responsive: true,
           plugins: {
               title: {
                   display: true,
                   text: 'Chart Title'
               },
               legend: {
                   position: 'top'
               }
           }
       }
   });
   </script>

Numbering HTML List Items from Zero

In certain web development scenarios, it becomes beneficial to display lists starting at zero rather than the default of one. This guide presents a concise, user-friendly approach to achieve such an effect through inline styling. The method offered below operates without external style sheets or embedded <style> sections. The following points illustrate an effective implementation.

Purpose of Starting Lists at Zero

• Facilitates indexing, particularly in contexts that rely on zero-based numbering (e.g., programming and data structures).
• Offers consistency when transitioning from programming languages, especially those that emphasize zero-based indexing.
• Highlights a custom design choice that can differentiate content.

Inline Styling Example

Below is a complete snippet demonstrating an ordered list beginning at zero:

<li><a href="#code">Code and Step-by-Step Explanation</a>
    <ol style="counter-reset: list-counter -1; list-style: none; padding-left: 0;">
        <li style="counter-increment: list-counter;">
            <span style="content: counter(list-counter) '. '; display: inline-block;">0. </span>
            <a href="#install-and-load">Install and Load Required Packages</a>
        </li>
        <li style="counter-increment: list-counter;">
            <span style="content: counter(list-counter) '. '; display: inline-block;">1. </span>
            <a href="#define-output">Define Output Directory</a>
        </li>
        <!-- Additional list items follow the same pattern -->
    </ol>
</li>

1. counter-reset
   Initializes the counter at -1, ensuring that the first increment displays as zero.
2. list-style: none; padding-left: 0;
   Removes default list styling and aligns the text as desired.
3. counter-increment: list-counter;
   Increments the list-counter for each list item (<li>).
4. Prefixed Numbers in <span>
   Inserts the actual numbers (e.g., 0., 1., etc.) in a <span> element. Inline CSS cannot dynamically display the counter value in the content property, so manual insertion of the incremented value is necessary.

Written on December 24th, 2024

Styling Blockquotes with Inline CSS

Blockquotes are essential elements in HTML used to denote sections of content that are quoted from another source. Applying custom styles enhances their visual distinction and improves readability. This guide provides a structured approach to styling blockquotes using inline CSS, eliminating the need for external style sheets or embedded <style> sections.

Purpose of Customizing Blockquotes

• Enhances the visual appeal and differentiation of quoted content.
• Improves readability by clearly distinguishing quotes from the main text.
• Allows for quick implementation without modifying external CSS files.
• Facilitates consistent styling across various sections of a webpage.

Inline Styling Example

The following snippet demonstrates how to style a blockquote with specific inline CSS properties:

<blockquote style="border-left: 4px solid #ccc; padding-left: 16px; color: #555; font-style: italic; margin-bottom: 20px;">
    This is an example of a styled blockquote. Inline CSS is used to apply custom styles directly to the element.
</blockquote>

1. border-left: 4px solid #ccc;
   Adds a left border with a width of 4 pixels, a solid style, and a light gray color (#ccc) to visually separate the blockquote from the surrounding content.
2. padding-left: 16px;
   Inserts 16 pixels of padding to the left side of the blockquote, ensuring that the text does not overlap with the left border and maintains adequate spacing.
3. color: #555;
   Sets the text color to a medium gray (#555), providing a subtle contrast against the default text color and enhancing readability.
4. font-style: italic;
   Applies an italic style to the text within the blockquote, traditionally used to emphasize quoted material.
5. margin-bottom: 20px;
   Adds a 20-pixel margin below the blockquote, creating space between the quote and subsequent content for better visual separation.

Written on December 5th, 2024

Click-to-Copy Implementation for Preformatted Blocks

Multiple approaches exist to implement a convenient "Click to Copy" feature for content enclosed within <pre> blocks. Two notable strategies—Version 1 and Version 3—rely on modern and traditional clipboard APIs, respectively. Both approaches incorporate a user-friendly button that copies the content directly to the clipboard with minimal effort.

Below is an integrated, refined explanation of these two methods, complete with annotated code snippets and concise notes on their operational differences. This writing maintains a formal perspective and aims to provide clarity, structure, and a professional overview.

Version 1: Single Function per Button (Modern Clipboard API)

This version attaches inline JavaScript to each copy button. When the button is clicked, the script accesses the adjacent <pre> element and calls navigator.clipboard.writeText(...) to copy its text. The built-in promise-based API confirms successful copying and can provide immediate visual feedback.

• Utilizes navigator.clipboard.writeText(...).
• Implements a simple inline function for each button's onclick handler.
• Offers modern browser compatibility.

    <!DOCTYPE html>
    <html lang="en">
    <head>
      <meta charset="UTF-8">
      <title>Version 1: Single Function per Button</title>
      <style>
        .copy-button {
          float: right;
          margin-top: 5px;
          cursor: pointer;
          background-color: #007bff;
          color: white;
          padding: 5px 10px;
          border: none;
          border-radius: 4px;
        }
        .copy-button:hover {
          background-color: #0056b3;
        }
        pre {
          position: relative;
          background-color: #f4f4f4;
          padding: 10px;
          border-radius: 5px;
          border: 1px solid #ddd;
          white-space: pre-wrap;
          margin-bottom: 0;
        }
      </style>
  
      <script>
        function copyToClipboard(button) {
          const preTag = button.previousElementSibling;
          const textToCopy = preTag.textContent;
          navigator.clipboard.writeText(textToCopy).then(() => {
            button.textContent = 'Copied!';
            setTimeout(() => { button.textContent = 'Copy'; }, 2000);
          });
        }
      </script>
    </head>
    <body>
    
      <h1>Version 1: Single Function per Button (Inline JS)</h1>
    
      <div style="width: 50%; margin: 0 auto;">
        <pre>
    This is some sample text 
    inside a preformatted block.
        </pre>
         <button class="copy-button" onclick="copyToClipboard(this)"> Copy </button>
      </div>
    
      <br>
    
      <div style="width: 50%; margin: 0 auto;">
        <pre>
    Another pre block 
    with more lines to copy.
        </pre>
        <button class="copy-button" onclick="copyToClipboard(this)">Copy</button>
      </div>
    
    </body>
    </html>

Version 2: Using a Hidden <textarea> and execCommand('copy')

This version uses the older but still viable document.execCommand('copy'). A hidden <textarea> is created and populated with the <pre> text, allowing selection and copying of the content. The <textarea> is removed afterward. This approach can enhance compatibility with certain legacy browsers.

• Employs document.execCommand('copy').
• Creates and removes a hidden <textarea> for copying.
• Can be a fallback for older browsers.

<!DOCTYPE html>
<html lang="en">
    <head>
      <meta charset="UTF-8">
      <title>Version 3: Using Hidden <textarea> and execCommand</title>
      <style>
        .copy-button {
          float: right;
          margin-top: 5px;
          cursor: pointer;
          background-color: #17a2b8;
          color: white;
          padding: 5px 10px;
          border: none;
          border-radius: 4px;
        }
        .copy-button:hover {
          background-color: #117a8b;
        }
        pre {
          position: relative;
          background-color: #f4f4f4;
          padding: 10px;
          border-radius: 5px;
          border: 1px solid #ddd;
          white-space: pre-wrap;
          margin-bottom: 0;
        }
      </style>  
    </head>
    <body>
    
      <h1>Version 2: Using Hidden <textarea> and execCommand</h1>
    
      <div style="width: 50%; margin: 0 auto;">
        <pre id="code-block-1">
    Sample text for Version 3,
    using hidden textarea.
        </pre>
        <button class="copy-button" onclick="copyWithExecCommand('code-block-1')"> Copy </button>
      </div>
    
      <br>
    
      <div style="width: 50%; margin: 0 auto;">
        <pre id="code-block-2">
    Another text block that 
    can be copied using execCommand.
        </pre>
        <button class="copy-button" onclick="copyWithExecCommand('code-block-2')">Copy</button>
      </div>
    
      <script>
        function copyWithExecCommand(preId) {
          const preElement = document.getElementById(preId);
          const textToCopy = preElement.textContent;
    
          // Create a hidden textarea element
          const textArea = document.createElement("textarea");
          textArea.value = textToCopy;
          textArea.style.position = "fixed";
          textArea.style.top = "-9999px";
          document.body.appendChild(textArea);
    
          // Select and copy
          textArea.select();
          try {
              document.execCommand("copy");
              // Visual feedback
              const button = preElement.nextElementSibling;
              button.textContent = "Copied!";
              setTimeout(() => { button.textContent = "Copy"; }, 2000);
          } catch (err) {
              console.error('Failed to copy: ', err);
              const button = preElement.nextElementSibling;
              button.textContent = "Error";
              setTimeout(() => { button.textContent = "Copy"; }, 2000);
          }
    
          // Cleanup
          document.body.removeChild(textArea);
        }
      </script> 
    </body>
</html>

Written on December 31th, 2024

Converting a Single-Column Table into a Two-Column Structure with Functional Copy Buttons

The following explanations summarize the modifications required to transform a one-column table layout into a two-column structure, ensuring that Copy buttons can accurately copy corresponding text from <pre> blocks. The new layout offers cleaner organization and more robust functionality, following a professional, hierarchical approach.

1. Table Structure Modifications

1. Separating <pre> Content and Copy Buttons

   • Each table row (<tr>) has been updated to contain two cells (<td>).
   • The first cell holds the <pre> block with the text to be copied, and the second cell contains the Copy button.

2. Ensuring Correct Alignment

   • The <pre> elements remain in the left column (<td>), while each corresponding Copy button is placed in the right column (<td>).
   • Proper vertical-align: top; ensures that the text and buttons are visually aligned.

3. HTML Example

   Below is a simplified illustration of the two-column structure:

   Be formal, without using I we you. Use humble tone. Make title. Based on these different version of writings above, additively, rewrite this into integrated refined writing. When you integrate, please do not remove thoughts and idea, but rearrange and develope further if applicable. I will give you flexibility to refine and upgrade the writing by rearranging thoughts and develop. Double check the writing and proof read before publishing I need systemic, hierarchical, comprehenstive, professional writing for publishing. If you could enhance the writing with bold-face, table, chart, and other illustrative format, please do so.

   Copy

2. Consolidated Copy Function

1. Combining the Copy Logic

   • A single function named copyToClipboard_enhanced replaces any previous functions, such as copyWithExecCommand, to avoid confusion.
   • This function employs the modern Clipboard API (navigator.clipboard) and ensures that the exact <pre> content from the same row is copied.

2. Locating the <pre> Element

   • The button uses button.closest('tr') to find its row and then tr.querySelector('pre') to locate the text targeted for copying.

3. User Feedback

   • The button text and styling briefly change to indicate successful copying or errors.
   • After two seconds, the button reverts to its original text and color.

4. Example Code

   <script>
       function copyToClipboard_enhanced(button) {
           const tr = button.closest('tr');
           if (!tr) {
               console.error('Copy button is not inside a table row.');
               return;
           }
   
           const pre = tr.querySelector('pre');
           if (!pre) {
               console.error('No <pre> element found in the same row as the copy button.');
               return;
           }
   
           const textToCopy = pre.textContent;
   
           navigator.clipboard.writeText(textToCopy).then(() => {
               const originalText = button.textContent;
               button.textContent = 'Copied!';
               button.style.backgroundColor = '#218838'; 
   
               setTimeout(() => {
                   button.textContent = originalText;
                   button.style.backgroundColor = '#28a745'; 
               }, 2000);
           }).catch(err => {
               console.error('Failed to copy text: ', err);
               button.textContent = 'Error';
               setTimeout(() => {
                   button.textContent = 'Copy';
                   button.style.backgroundColor = '#28a745'; 
               }, 2000);
           });
       }
   </script>

3. Additional Steps and Considerations

1. Removing Redundant Functions

   • Any old functions (copyWithExecCommand) should be removed to prevent conflicts with the new consolidated function.

2. Fallback Support

   • To support older browsers that do not recognize navigator.clipboard, consider adding a fallback using document.execCommand('copy').

3. Styling and Responsiveness

   • A .simple-table class can maintain consistent spacing and borders.
   • Media queries (e.g., @media (max-width: 600px)) can convert the two-column layout into a stacked view on smaller screens.

4. Accessibility

   • Adding aria-label to the Copy button can improve screen-reader compatibility:

   <button class="copy-button" onclick="copyToClipboard_enhanced(this)" aria-label="Copy content">Copy</button>

5. Proofreading and Testing

   • Verifying that each button correctly copies the intended <pre> block.
   • Checking for console errors in the browser’s developer tools.
   • Ensuring that the text is copied properly by pasting into an external editor.

4. Example of the Updated Layout

Below is a concise illustration of the table and script for reference:

Sample Text for Copy

<script>
    function copyToClipboard_enhanced(button) {
        // Consolidated copy function as described above
    }
</script>

Written on December 31th, 2024

Dotted Border Box Styling Options

Inline CSS can be used to create a box with a dotted border. One common approach is to use the style attribute with properties such as border, padding, and border-radius. The following examples show various color options for a dotted border box.

1. Bare-bone example

   <div style="border: 3px dotted #66b266; padding: 10px; border-radius: 5px;">
     Your content here.
   </div>
   

   • border: 1px dotted #66b266; – Applies a 1 pixel dotted border with the color #66b266.
   • padding: 10px; – Provides spacing between the border and the content.
   • border-radius: 5px; – Rounds the corners of the border.

   Below are additional color options for the dotted border box:

   Option	Color Code	Example
   Option 1	#66b266 (Medium Green)	Content here.
   Option 2	#008000 (Dark Green)	Content here.
   Option 3	#006400 (Very Dark Green)	Content here.
   Option 4	#2e8b57 (Sea Green)	Content here.

   • Each option uses the same padding and border-radius values to maintain consistency in layout.
   • The border color can be easily adjusted by changing the hexadecimal color value in the border property.
   • These inline styles provide a quick method to style elements without requiring external CSS.

2. Bare-bone example with Label

   <div style="position: relative; border: 3px dotted #66b266; padding: 20px; border-radius: 5px; margin: 20px 0;">
     <span style="position: absolute; top: -0.75em; left: 12px; background: #fff; padding: 0 6px; font-weight: bold;">
       Your Label Here
     </span>
   
     <ol>
       <li>First item...</li>
       <li>Second item...</li>
       <!-- Add more items as needed -->
     </ol>
   </div>
   

Written on March 14, 2025

Collapsible Menu with <details> and <summary>

The <details> and <summary> elements provide a simple way to create collapsible or expandable sections. When the user clicks the <summary> (the visible heading), the associated content in <details> toggles between hidden and shown. This is helpful for creating “tear-down” menus or sections that can be revealed on demand.

1. Bare-bones example

   <details>
     <summary></summary>
   </details>
   

   • <details>: Wraps the entire collapsible section. By default, the content inside <details> is hidden until the user interacts with it.
   • <summary>: The visible heading or label that the user can click to toggle the visibility of the content within the <details> element.
   • open attribute (optional): If you add open to <details>, the content will be shown by default:
     <details open><summary>...</summary>...</details>

2. Example with inline <h3> inside <summary>

   The following variant places an <h3> directly inside <summary> (kept on a single line for cleaner rendering) and shows how to leave an in-code comment that can be inspected from developer tools:

   <details> 
     <summary><h3 style="display:inline; margin:0;">Console demo</h3></summary>
     <p>
       Open your browser’s console and type $0 to inspect this
       <details> element instance.
     </p>
   </details>
   

   • <h3> inline style: Using display:inline; keeps the heading on the same line as the triangle icon, avoiding unwanted line breaks.

3. Customized example with margins, padding, and list

   Here’s a slightly more customised example that adds inline CSS for spacing and cursors:

   <details style="margin: 10px 0; padding: 10px; border: 1px solid #ccc;">
     <summary style="font-size:1.5em; font-weight:bold;">Toggle Menu</summary>
     <p>Details shown after toggling.</p>
     <ul>
       <li>Item 1</li>
       <li>Item 2</li>
       <li>Item 3</li>
     </ul>
   </details>
   

   • style="margin: 10px 0; padding: 10px; border: 1px solid #ccc;": Adds spacing, padding, and a subtle border to separate this collapsible block from other content on the page.
   • summary style="cursor:pointer; font-weight:bold;": Ensures the mouse pointer changes when hovering over the summary (making it clear it can be clicked), and makes the summary text bold.

   With <details> and <summary>, content can remain tucked away until requested, keeping pages organised and improving the overall user experience.

4. Tiny JavaScript helper: show <details> on hover AND click

   Sometimes you want a <details> panel to reveal itself when the mouse merely hovers over its <summary>, while still preserving keyboard/click accessibility.
   The snippet below adds that behaviour to every <details class="hoverable"> on the page.

   <details class="hoverable">
     <summary>I open on hover (and stay on click)</summary>
     <p>Content that appears while hovered or when pinned open.</p>
   </details>
   
   <script>
     document.querySelectorAll('details.hoverable').forEach(det => {
       const show  = () => det.setAttribute('open', '');
       const hide  = () => det.removeAttribute('open');
       det.addEventListener('mouseenter', show);   // hover in
       det.addEventListener('mouseleave', hide);   // hover out
     });
   </script>
   

   • Add class="hoverable" only to panels that should support hover reveal, leaving others to act normally.
   • Accessibility-first: the native <details> semantics remain untouched, so screen-readers and keyboard users still get predictable behaviour.
   • Graceful fallback: if JavaScript is disabled, the panel reverts to standard click-to-toggle, preserving content access.

5. Resetting styles with inline style="all: revert;"

   If you’ve applied global or scoped CSS rules to <details> and need a particular instance to ignore those rules, add style="all: revert;". This resets every property on that element to the user-agent (browser) default—or to the next-highest rule—giving you a clean slate for further inline styling.

   <!-- A styled details panel that inherits page-level styles -->
   <details style="margin:10px 0; padding:10px; border:1px solid #ccc; background:#f9f9f9;">
     <summary>Styled panel</summary>
     <p>This panel inherits the page’s styles.</p>
   </details>
   
   <!-- A reset panel that ignores those same page-level styles -->
   <details style="all: revert;">
     <summary>Reset panel</summary>
     <p>This panel ignores any global <details> rules and shows browser defaults.</p>
   </details>
   

   • all: revert;: Clears previously applied styles (including margins, borders, fonts, etc.) for this element, reverting them to the browser default or inherited values.
   • Order matters: Any additional inline properties written after all: revert; will override the reverted values, letting you selectively restyle only what you need.
   • Browser support: Fully supported in all evergreen browsers (Chrome ≥96, Firefox ≥97, Safari ≥15.4, Edge ≥96). Older browsers may ignore the declaration.

Written on March 29, 2025

No Indent Paragraph

In some cases, you may want to ensure that a paragraph has no first-line indentation while still keeping space between paragraphs. This can be achieved using inline CSS on the <p> tag. The following code demonstrates how to remove indentation while preserving paragraph spacing.

<p style="margin:0 0 1em 0; text-indent:0;">
  Your paragraph text goes here.
</p>

• margin:0 0 1em 0;: Removes any top margin, keeps a 1em bottom margin to separate paragraphs, and sets left/right margins to 0.
• text-indent:0;: Ensures there is no indentation on the first line of the paragraph.

You can also simply use:

<p style="text-indent:0;">
  Your paragraph text goes here.
</p>

Written on September 29, 2025

Styling omitted manuscript sections with inline CSS (Written December 29, 2025)

In technical and academic documents, certain sections may be intentionally retained for archival transparency while being visually marked as excluded from later editions. Inline CSS provides a precise and self-contained method to annotate such regions without relying on external stylesheets or embedded <style> blocks. The following example demonstrates a structured approach to visually denoting omitted content while preserving full semantic HTML structure.

Purpose of visually marking omitted sections

• Clearly communicates editorial intent without deleting original content.
• Preserves reproducibility and version traceability in technical manuscripts.
• Visually de-emphasizes deprecated material while keeping it readable.
• Allows localized styling without global CSS side effects.

Inline styling example

The snippet below illustrates an inline-styled container that applies opacity reduction, a diagonal hatch overlay, and an annotation label indicating omission status:

<div style="margin-left: 3em; position: relative; opacity: 0.65;">

  <!-- overlay -->
  <div style="
    position: absolute;
    inset: 0;
    background:
      repeating-linear-gradient(
        135deg,
        rgba(0,0,0,0.08),
        rgba(0,0,0,0.08) 10px,
        rgba(0,0,0,0.00) 10px,
        rgba(0,0,0,0.00) 20px
      );
    pointer-events: none;
    z-index: 2;
  "></div>

  <!-- label -->
  <div style="
    position: absolute;
    top: 6px;
    right: 10px;
    font-size: 0.85rem;
    color: #555;
    background: rgba(255,255,255,0.9);
    padding: 2px 6px;
    border: 1px dashed #999;
    z-index: 3;
  ">
    Omitted in later edition
  </div>

~~~~~

</div>

Explanation of key styling components

1. position: relative;
   Establishes the container as a positioning context, enabling precise absolute placement of overlays and labels within the omitted section.
2. opacity: 0.65;
   Reduces visual prominence of the enclosed content while maintaining legibility, signaling secondary or deprecated status.
3. repeating-linear-gradient(...)
   Generates a diagonal hatch pattern commonly associated with exclusions or placeholders, reinforcing the notion of omission through visual language.
4. pointer-events: none;
   Ensures that the overlay does not interfere with text selection, scrolling, or interactive elements contained within the section.
5. z-index layering
   Separates content (z-index: 1, implicit), overlay (z-index: 2), and label (z-index: 3) to maintain clarity and proper stacking order.
6. Annotation label styling
   The dashed border, muted color palette, and semi-opaque background create a non-intrusive yet unambiguous editorial marker.

This pattern is particularly suitable for draft manuscripts, preprints, and technical appendices where transparency of evolution is valued. Because all styling is inline, the construct remains portable across platforms, exports, and static HTML distributions.

Written on December 29, 2025

Images

Inline responsive two‑row image gallery templates

Two minimal, inline‑styled patterns enable quick assembly of three‑image galleries inside a fixed‑width page (65 em in the parent example). Version I covers layouts with three landscape images. Version II extends the technique to a mix of one portrait and two landscape images, ensuring equal height on the second row through lightweight JavaScript.

Version I — three landscape images

Concept

• The first row hosts one landscape image at full container width.
• The second row aligns two landscape images side by side, each at 50 % width.
• No JavaScript is required, as equal heights are not enforced; the images’ intrinsic aspect ratios remain intact via height:auto.

Key attributes

Code snippet

<div style="width: 100%;">
  <img src="src/20250506_Yamaha02.jpeg"
       alt="Yamaha 02"
       style="width: 100%; height: auto;">
</div>

<div style="display: flex; width: 100%; gap: 8px;">
  <img src="src/20250506_Yamaha01.jpeg"
       alt="Yamaha 01"
       style="width: 50%; height: auto;">
  <img src="src/20250506_Yamaha03.jpeg"
       alt="Yamaha 03"
       style="width: 50%; height: auto;">
</div>

Version II — one portrait, two landscape images

Concept

• The first row mirrors Version I with a full‑width landscape image.
• The second row houses one portrait and one landscape image. A script reads intrinsic aspect ratios, calculates a common height, then applies widths so both images plus the gap fill the row flawlessly.
• ResizeObserver (with a window.resize fallback) ensures constant responsiveness.

Key attributes

Code snippet

<!-- parent layout (65 em body and .main block) shown in Version I -->
<div style="display:grid;grid-template-rows:auto auto;gap:8px;width:100%;max-width:100%;">
  <!-- Row 1 -->
  <div>
    <img id="img02" src="/src/20250506_glock18_02.jpeg"
         alt="Landscape 02"
         style="max-width:100%;height:auto;display:block;">
  </div>

  <!-- Row 2 -->
  <div id="row2" style="display:flex;gap:8px;width:100%;max-width:100%;">
    <img id="img01" src="/src/20250506_glock18_01.jpeg"
         alt="Portrait 01"
         style="display:block;max-width:100%;height:auto;">
    <img id="img03" src="/src/20250506_glock18_03.jpeg"
         alt="Landscape 03"
         style="display:block;max-width:100%;height:auto;">
  </div>
</div>

<script>
(function () {
  const img1 = document.getElementById('img01');
  const img3 = document.getElementById('img03');
  const row2 = document.getElementById('row2');

  function resizeRow() {
    const gap = parseFloat(getComputedStyle(row2).gap) || 0;
    const total = row2.clientWidth - gap;
    const r1 = img1.naturalWidth / img1.naturalHeight;
    const r3 = img3.naturalWidth / img3.naturalHeight;
    if (!r1 || !r3) return;
    const h = total / (r1 + r3);
    img1.style.height = img3.style.height = h + 'px';
    img1.style.width  = (r1 * h) + 'px';
    img3.style.width  = (r3 * h) + 'px';
  }

  function onBothLoaded(cb) {
    let ready = 0;
    [img1, img3].forEach(img => {
      if (img.complete) ready++;
      else img.addEventListener('load', () => { if (++ready === 2) cb(); });
    });
    if (ready === 2) cb();
  }

  function observe() {
    if ('ResizeObserver' in window) {
      new ResizeObserver(resizeRow).observe(row2);
    } else {
      window.addEventListener('resize', resizeRow);
    }
  }

  onBothLoaded(() => { resizeRow(); observe(); });
})();
</script>

Troubleshooting tips

• Add max-width:100%; on every image to prevent unintended up‑scaling.
• Verify that parent paddings or margins do not extend beyond the 65 em constraint.
• On older browsers, confirm that the window.resize fallback triggers as expected.

Written on May 6, 2025

Lightbox implementation guide 🔍

Ⅰ. Purpose

A concise reference for re‑using the pure‑CSS / vanilla‑JS lightbox that enables:

• full‑width thumbnail images (aspect‑ratio preserved)
• overlay popup of the original image on click
• dismissal via ✕ button, overlay click, image click, or Esc key

Ⅱ. Modular structure

Ⅲ. Markup snippet (HTML)

<!-- Thumbnail wrapped in anchor -->
<a href="assets/photo01_large.jpg" class="lightbox">
  <img src="assets/photo01_thumb.jpg"
       alt="Descriptive alt text"
       style="width:100%; height:auto; cursor:zoom-in;">
</a>

Place additional images by repeating the same pattern; the script will discover every .lightbox link automatically.

Ⅳ. Stylesheet (lightbox.css)

/* Overlay container – hidden until activated */
#lightbox-overlay {
  position: fixed;
  inset: 0;
  background: rgba(0, 0, 0, 0.65);
  display: none;           /* toggled via JS */
  align-items: center;
  justify-content: center;
  z-index: 9998;
}

/* Enlarged image inside overlay */
#lightbox-overlay img {
  max-width: 90vw;
  max-height: 90vh;
  box-shadow: 0 0 10px rgba(0, 0, 0, 0.6);
  cursor: zoom-out;
}

/* Close (✕) button */
#lightbox-overlay .close-btn {
  position: absolute;
  top: 20px;
  right: 30px;
  font: 700 2rem/1 sans-serif;
  color: #fff;
  cursor: pointer;
}

Kindly save as lightbox.css and link once in the <head> section.

Ⅴ. Script (lightbox.js)

/* Lightbox – reusable on any page containing <a class="lightbox"> links */
(function () {
  // Build overlay once DOM is ready
  document.addEventListener('DOMContentLoaded', () => {
    const overlay = document.createElement('div');
    overlay.id = 'lightbox-overlay';
    overlay.innerHTML = `
      <span class="close-btn" aria-label="Close lightbox">✕</span>
      <img src="" alt="">
    `;
    document.body.appendChild(overlay);

    const imgTag   = overlay.querySelector('img');
    const closeBtn = overlay.querySelector('.close-btn');

    const open = src => {
      imgTag.src = src;
      overlay.style.display = 'flex';
    };
    const close = () => {
      overlay.style.display = 'none';
      imgTag.src = '';
    };

    /* Activate every anchor with class="lightbox" */
    document.querySelectorAll('a.lightbox').forEach(link => {
      link.addEventListener('click', e => {
        e.preventDefault();
        open(link.href);
      });
    });

    /* Close handlers */
    closeBtn.addEventListener('click', close);
    overlay.addEventListener('click', e => {
      if (e.target === overlay || e.target === imgTag) close();
    });
    document.addEventListener('keyup', e => {
      if (e.key === 'Escape' && overlay.style.display === 'flex') close();
    });
  });
})();

Link the script immediately before the closing </body> tag for best performance.

Ⅵ. Integration checklist ✅

1. Add markup – wrap each thumbnail in an <a class="lightbox">.
2. Include stylesheet – <link rel="stylesheet" href="lightbox.css">.
3. Include script – <script src="lightbox.js"></script> before </body>.
4. Verify – click a thumbnail: overlay appears; click ✕, the image, the background, or press Esc → overlay closes.

This modular arrangement permits effortless re‑use across projects while keeping markup clean and maintenance minimal.

Written on May 7, 2025

Inline image layout templates

An aide-mémoire for arranging images with pure inline styles, relying on flexbox for simple horizontal alignment.

Key points

• display:flex on the parent aligns child elements in a row.
• gap applies consistent spacing without extra wrappers.
• flex:1 on each <li> (or equivalent) distributes width evenly.
• Reset list styling with list-style:none; padding:0; margin:0; when using <ul>.
• Add vertical rhythm between rows by adjusting the parent’s margin-top.

Code snippets

Single-image block (1 × 1)

<a href="/src/20250514_hades2.png" class="lightbox">
  <img src="/src/20250514_hades2.png"
       alt="Descriptive alt text"
       style="width:100%; height:auto; cursor:zoom-in;">
</a>

One row, two images (1 × 2)

<ul style="display:flex; list-style:none; padding:0; margin:0; gap:10px;">
  <li style="flex:1;">
    <a href="/src/20250515_hades20.png" class="lightbox">
      <img src="/src/20250515_hades20.png"
           alt="Descriptive alt text"
           style="width:100%; height:auto; cursor:zoom-in;">
    </a>
  </li>
  <li style="flex:1;">
    <a href="/src/20250515_hades21.png" class="lightbox">
      <img src="/src/20250515_hades21.png"
           alt="Descriptive alt text"
           style="width:100%; height:auto; cursor:zoom-in;">
    </a>
  </li>
</ul>

Three rows, two images each (3 × 2)

<!-- Row 1 -->
<ul style="display:flex; list-style:none; padding:0; margin:0; gap:10px;">
  <li style="flex:1;">
    <a href="/src/20250515_hades100.png" class="lightbox">
      <img src="/src/20250515_hades100.png"
           alt="Descriptive alt text"
           style="width:100%; height:auto; cursor:zoom-in;">
    </a>
  </li>
  <li style="flex:1;">
    <a href="/src/20250515_hades101.png" class="lightbox">
      <img src="/src/20250515_hades101.png"
           alt="Descriptive alt text"
           style="width:100%; height:auto; cursor:zoom-in;">
    </a>
  </li>
</ul>

<!-- Row 2 -->
<ul style="display:flex; list-style:none; padding:0; margin:10px 0 0 0; gap:10px;">
  <li style="flex:1;">
    <a href="/src/20250515_hades102.png" class="lightbox">
      <img src="/src/20250515_hades102.png"
           alt="Descriptive alt text"
           style="width:100%; height:auto; cursor:zoom-in;">
    </a>
  </li>
  <li style="flex:1;">
    <a href="/src/20250515_hades103.png" class="lightbox">
      <img src="/src/20250515_hades103.png"
           alt="Descriptive alt text"
           style="width:100%; height:auto; cursor:zoom-in;">
    </a>
  </li>
</ul>

<!-- Row 3 -->
<ul style="display:flex; list-style:none; padding:0; margin:10px 0 0 0; gap:10px;">
  <li style="flex:1;">
    <a href="/src/20250515_hades104.png" class="lightbox">
      <img src="/src/20250515_hades104.png"
           alt="Descriptive alt text"
           style="width:100%; height:auto; cursor:zoom-in;">
    </a>
  </li>
  <li style="flex:1;">
    <a href="/src/20250515_hades105.png" class="lightbox">
      <img src="/src/20250515_hades105.png"
           alt="Descriptive alt text"
           style="width:100%; height:auto; cursor:zoom-in;">
    </a>
  </li>
</ul>

<!-- Gallery block -->
<div style="display:grid; grid-template-rows:auto auto; gap:8px; width:100%; max-width:100%;">

  <!-- Row 1 – full-width landscape -->
  <a href="/src/20250506_glock18_02.jpeg" class="lightbox">
    <img src="/src/20250506_glock18_02.jpeg"
         alt="Descriptive alt text"
         style="display:block; max-width:100%; height:auto; cursor:zoom-in;">
  </a>

  <!-- Row 2 – portrait + landscape matched by height -->
  <div class="match-height-row" style="display:flex; gap:8px; width:100%; max-width:100%;">
    <a href="/src/20250506_glock18_01.jpeg" class="lightbox">
      <img src="/src/20250506_glock18_01.jpeg"
           alt="Descriptive alt text"
           style="display:block; max-width:100%; height:auto; cursor:zoom-in;">
    </a>
    <a href="/src/20250506_glock18_03.jpeg" class="lightbox">
      <img src="/src/20250506_glock18_03.jpeg"
           alt="Descriptive alt text"
           style="display:block; max-width:100%; height:auto; cursor:zoom-in;">
    </a>
  </div>

</div>

<script>
/*  Ensures every .match-height-row with exactly two <img> children
    displays them at equal height while respecting their aspect ratios. */
(function () {
  function adjust(row) {
    const imgs = Array.from(row.querySelectorAll('img'));
    if (imgs.length !== 2) return;                       // supports pairs only
    const gap   = parseFloat(getComputedStyle(row).gap) || 0;
    const width = row.clientWidth - gap;
    const ratios = imgs.map(img => img.naturalWidth / img.naturalHeight);
    if (ratios.some(r => !r)) return;                   // images not ready
    const h = width / (ratios[0] + ratios[1]);           // common height
    imgs.forEach((img, i) => {
      img.style.height = h + 'px';
      img.style.width  = (ratios[i] * h) + 'px';
    });
  }

  function init(row) {
    const imgs = row.querySelectorAll('img');
    let loaded = 0;
    imgs.forEach(img => {
      if (img.complete) loaded++;
      else img.addEventListener('load', () => { if (++loaded === imgs.length) adjust(row); });
    });
    if (loaded === imgs.length) adjust(row);
    new ResizeObserver(() => adjust(row)).observe(row);
  }

  document.querySelectorAll('.match-height-row').forEach(init);
})();
</script>

Two images, equal height – dynamic inline JS (update YYYYMMDD prefix)

Whenever this template is copied, replace every instance of YYYYMMDD in the id attributes with today’s date. That simple habit keeps all IDs unique and avoids namespace collisions.

<!-- -------------------------------------------------------------------- -->
<!-- Quick reminder: update the ID prefix each time you reuse this        -->
<!-- Replace YYYYMMDD with today’s date (e.g., 20250810) so IDs           -->
<!-- stay unique and never clash with other galleries on the page.        -->
<!-- Whenever this template is copied, replace every instance of          -->
<!-- YYYYMMDD in the id attributes with today’s date.                     -->
<!-- -------------------------------------------------------------------- -->

<figure id="YYYYMMDD_gallery"
        style="display:flex; gap:10px; margin:1rem 0; align-items:flex-start; flex-wrap:wrap;">
  <!-- Left (landscape) image -->
  <a href="/src/20250721_resmed05.jpg" class="lightbox">
    <img id="YYYYMMDD_imgLeft"
         src="/src/20250721_resmed05.jpg"
         alt="Blue pressure-sense tube on ResMed Astral circuit"
         style="display:block; width:auto; height:auto; cursor:zoom-in;">
  </a>

  <!-- Right (portrait) image -->
  <a href="/src/20250801_resmed01.jpg" class="lightbox">
    <img id="YYYYMMDD_imgRight"
         src="/src/20250801_resmed01.jpg"
         alt="Alternate view of blue pressure-sense tube on ResMed circuit"
         style="display:block; width:auto; height:auto; cursor:zoom-in;">
  </a>

  <!-- Caption spans full width below the images -->
  <figcaption style="flex-basis:100%; text-align:center; font-size:0.9rem; margin-top:0.5rem;">
    The blue tube is for pressure, which is the way to determine whether it is a check valve or a leak valve.
  </figcaption>
</figure>

<script>
  (function () {
    /* Dynamically equalize heights and keep total width within figure */
    function resizeDualImages () {
      const fig   = document.getElementById('YYYYMMDD_gallery');
      const left  = document.getElementById('YYYYMMDD_imgLeft');
      const right = document.getElementById('YYYYMMDD_imgRight');
      if (!left.complete || !right.complete) return;          /* wait until both are loaded */

      const gapPx = 10;                                       /* inline gap above */
      const cw    = fig.clientWidth;

      const arL = left.naturalWidth  / left.naturalHeight;    /* aspect ratios */
      const arR = right.naturalWidth / right.naturalHeight;

      let h = Math.min(left.naturalHeight, right.naturalHeight);  /* avoid up-scaling */

      let wNeeded = arL * h + arR * h + gapPx;                /* width needed at that height */

      if (wNeeded > cw) {                                     /* scale down if too wide */
        const scale = (cw - gapPx) / (arL * h + arR * h);
        h *= scale;
      }

      left.style.height  = h + 'px';
      left.style.width   = (arL * h) + 'px';
      right.style.height = h + 'px';
      right.style.width  = (arR * h) + 'px';
    }

    window.addEventListener('load',  resizeDualImages);
    window.addEventListener('resize', resizeDualImages);
  })();
</script>

Three images in one row — equal height, dynamic inline JS (update YYYYMMDD prefix; fully encapsulated to avoid namespace conflicts)

When you copy this template, replace every instance of YYYYMMDD in the id attributes with today’s date (e.g., 20251011). That simple habit keeps all IDs unique and prevents collisions if you place multiple galleries on the same page. The script is wrapped in an IIFE so no globals leak.

<!-- -------------------------------------------------------------------- -->
<!-- Quick reminder: update the ID prefix each time you reuse this        -->
<!-- Replace YYYYMMDD with today’s date (e.g., 20251011) so IDs           -->
<!-- stay unique and never clash with other galleries on the page.        -->
<!-- Whenever this template is copied, replace every instance of          -->
<!-- YYYYMMDD in the id attributes with today’s date.                     -->
<!-- -------------------------------------------------------------------- -->

<figure id="YYYYMMDD_gallery"
        style="display:flex; gap:10px; margin:1rem 0; align-items:flex-start; flex-wrap:nowrap;">

  <!-- Left (landscape) -->
  <a href="/src/REPLACE_leftLandscape.jpg" class="lightbox">
    <img id="YYYYMMDD_imgLeft"
         src="/src/REPLACE_leftLandscape.jpg"
         alt="Left landscape view"
         style="display:block; width:auto; height:auto; cursor:zoom-in;">
  </a>

  <!-- Center (portrait) -->
  <a href="/src/REPLACE_portrait.jpg" class="lightbox">
    <img id="YYYYMMDD_imgCenter"
         src="/src/REPLACE_portrait.jpg"
         alt="Center portrait view"
         style="display:block; width:auto; height:auto; cursor:zoom-in;">
  </a>

  <!-- Right (landscape) -->
  <a href="/src/REPLACE_rightLandscape.jpg" class="lightbox">
    <img id="YYYYMMDD_imgRight"
         src="/src/REPLACE_rightLandscape.jpg"
         alt="Right landscape view"
         style="display:block; width:auto; height:auto; cursor:zoom-in;">
  </a>

  <!-- Caption spans full width below the images -->
  <figcaption style="flex-basis:100%; text-align:center; font-size:0.9rem; margin-top:0.5rem;">
    Three images aligned to equal height: a portrait centered between two landscapes.
  </figcaption>
</figure>

<script>
(function gallery_YYYYMMDD() {
  /* Everything is scoped to this IIFE with a unique name to avoid namespace conflicts */
  const fig    = document.getElementById('YYYYMMDD_gallery');
  const left   = document.getElementById('YYYYMMDD_imgLeft');
  const center = document.getElementById('YYYYMMDD_imgCenter');
  const right  = document.getElementById('YYYYMMDD_imgRight');

  if (!fig || !left || !center || !right) return;

  function resizeTripleImages() {
    // Wait until all images are loaded
    if (!left.complete || !center.complete || !right.complete) return;

    const singleGap = 10;            // Match the gap in the figure style
    const gapPx     = singleGap * 2; // Two gaps between three images
    const cw        = fig.clientWidth;

    // Aspect ratios
    const arL = left.naturalWidth   / left.naturalHeight;
    const arC = center.naturalWidth / center.naturalHeight;
    const arR = right.naturalWidth  / right.naturalHeight;

    // Start from smallest natural height to avoid upscaling
    let h = Math.min(left.naturalHeight, center.naturalHeight, right.naturalHeight);

    // Calculate total width needed
    let wNeeded = arL * h + arC * h + arR * h + gapPx;

    // Scale down if total width exceeds container
    if (wNeeded > cw) {
      const scale = (cw - gapPx) / (arL * h + arC * h + arR * h);
      h *= scale;
    }

    // Apply explicit sizes
    left.style.height   = h + 'px';
    left.style.width    = (arL * h) + 'px';
    center.style.height = h + 'px';
    center.style.width  = (arC * h) + 'px';
    right.style.height  = h + 'px';
    right.style.width   = (arR * h) + 'px';
  }

  // Resize on load and on window resize
  window.addEventListener('load', resizeTripleImages);
  window.addEventListener('resize', resizeTripleImages);

  // Also bind resize on individual image load in case of delayed loading
  [left, center, right].forEach(img => {
    if (!img.complete) {
      img.addEventListener('load', resizeTripleImages, { once: true });
    }
  });
})();
</script>

Written on May 15, 2025

Videos

Comparing Video Modals and Embedded Videos

Video modals and embedded videos represent two distinct approaches to presenting video content within a webpage. While both rely on the HTML <video> element to display media, their methods of delivery, user interaction, and overall user experience differ significantly. Understanding these differences and learning how to implement each method enables more effective decision-making and design strategies.

Defining Embedded Videos and Video Modals

1. Embedded Videos

   Embedded videos are placed directly into the page’s layout, appearing among other textual or visual elements. They are immediately visible, often encouraging viewers to interact with them as part of the normal scrolling experience. This simplicity allows quick access but may result in less focused attention if the page content is dense.

   • Seamless integration into the page layout.
   • Always visible without additional user actions.
   • Straightforward HTML structure, often just a <video> element with sources.
   • Can be styled and resized using inline styles or CSS.
   • Suitable for scenarios where continuous access to the video is desired.

2. Video Modals

   Video modals present videos in an overlay that appears above the page’s main content when triggered. This approach reduces distractions by dimming background elements and focusing the viewer’s attention on the video. Although it requires more scripting and user interaction, it can provide a more immersive viewing experience.

   • Displayed in a pop-up overlay window.
   • Dims or blurs background content, enhancing focus on the video.
   • Requires user action (e.g., clicking a button) to open and close.
   • Provides a controlled viewing environment.
   • Ideal for promotional content, featured media, or highlighting special video assets.

Illustrative Code Examples

1. Embedded Video Example

   The following code snippet demonstrates a basic embedded video. The video is part of the page flow, using inline styling to ensure responsive sizing. This example can be expanded with multiple sources, captions, and attributes as needed.

   <!-- Embedded Video Example -->
   <video style="width:100%; height:auto;" controls>
     <source src="videos/example.mp4" type="video/mp4">
     <source src="videos/example.webm" type="video/webm">
     <!-- Optional Caption Track -->
     <track kind="captions" src="videos/captions.vtt" srclang="en" label="English">
     Your browser does not support the video tag.
   </video>
   

   • controls: Provides playback controls (play, pause, volume, fullscreen).
   • Multiple <source> elements: Ensures compatibility with different browsers.
   • <track> element: Adds captions for improved accessibility.
   • Inline styles: Manages responsive sizing without external CSS interference.

2. Video Modal Example

   Below is a sample video modal implementation. This example uses a trigger button to open a modal overlay containing the video, ensuring user focus remains on the media when it plays.

   <!-- Trigger Button -->
   <button id="openModal" style="padding:10px 20px; font-size:16px;">Watch Video</button>
   
   <!-- Modal Structure -->
   <div id="videoModal" style="
     display:none; position:fixed; z-index:1000; 
     left:0; top:0; width:100%; height:100%; 
     overflow:auto; background-color:rgba(0,0,0,0.8);
   " aria-hidden="true" role="dialog" aria-labelledby="modalTitle">
   
     <!-- Modal Content -->
     <div style="
       position:relative; margin:5% auto; padding:0; 
       width:80%; max-width:700px;
     ">
       <!-- Close Button -->
       <span id="closeModal" style="
         position:absolute; top:10px; right:25px; 
         color:#fff; font-size:35px; font-weight:bold; 
         cursor:pointer;
       ">&times;</span>
   
       <!-- Video in Modal -->
       <video style="width:100%; height:auto;" controls>
         <source src="videos/example.mp4" type="video/mp4">
         <track kind="captions" src="videos/captions.vtt" srclang="en" label="English">
         Your browser does not support the video tag.
       </video>
   
       <!-- Caption -->
       <figcaption style="color:#fff; text-align:center; margin-top:10px;">
         <em>Example video demonstrating modal overlay and inline styling.</em>
       </figcaption>
     </div>
   </div>
   
   <!-- JavaScript for Modal Functionality -->
   <script>
     const modal = document.getElementById('videoModal');
     const openBtn = document.getElementById('openModal');
     const closeBtn = document.getElementById('closeModal');
   
     openBtn.onclick = function() {
       modal.style.display = 'block';
     }
   
     closeBtn.onclick = function() {
       modal.style.display = 'none';
       const video = modal.querySelector('video');
       video.pause();
       video.currentTime = 0;
     }
   
     window.onclick = function(event) {
       if (event.target === modal) {
         modal.style.display = 'none';
         const video = modal.querySelector('video');
         video.pause();
         video.currentTime = 0;
       }
     }
   
     // Optional: Close modal with Esc key
     document.addEventListener('keydown', function(event) {
       if (event.key === 'Escape') {
         modal.style.display = 'none';
         const video = modal.querySelector('video');
         video.pause();
         video.currentTime = 0;
       }
     });
   </script>
   

   • Trigger Button: Opens the modal on click, displaying the overlay and the video.
   • Modal Structure: Uses position:fixed, width:100%, and height:100% to cover the entire viewport. The semi-transparent background (rgba(0,0,0,0.8)) dims the main content.
   • Close Button & Event Handling: The × icon, clicking outside the video area, or pressing the Esc key closes the modal. Upon closing, the script pauses and resets the video.
   • ARIA Attributes: role="dialog", aria-hidden="true", and aria-labelledby="modalTitle" enhance accessibility by providing context for assistive technologies.

Written on December 20th, 2024

Video Modal

Implementing an Automatic Video Modal with Adjustable Width

Creating an engaging user experience often involves presenting multimedia content in an accessible and aesthetically pleasing manner. This guide outlines the process of implementing an automatic pop-up (modal) window that displays a descriptive paragraph alongside an autoplaying video. The video is centered within the modal and defaults to a width of 70%, with the ability for users to adjust its size via a slider. The implementation ensures responsiveness and accessibility across various devices and browsers.

1. Structure the HTML

   Begin by setting up the HTML structure, including the modal container, content, paragraph, slider, and video elements.

   <!DOCTYPE html>
   <html lang="en">
   <head>
       <meta charset="UTF-8">
       <meta http-equiv="X-UA-Compatible" content="IE=edge">
       <meta name="viewport" content="width=device-width, initial-scale=1.0">
       <title>iOS App Demo Video</title>
       <!-- CSS will be added here -->
   </head>
   <body>
   
       <!-- Modal Structure -->
       <div id="video-modal" class="modal">
           <div class="modal-content">
               <span class="close">&times;</span>
               
               <!-- Descriptive Paragraph -->
               <p class="demo-paragraph">
                   This is a demo video for the upcoming iOS app, currently experiencing technical issues with submission to the Apple App Store.
               </p>
               
               <!-- Slider for Adjusting Video Width -->
               <div class="slider-container">
                   <label for="width-slider">Adjust video width: <span id="slider-value">70%</span></label>
                   <input 
                       type="range" 
                       id="width-slider" 
                       min="50" 
                       max="100" 
                       value="70" 
                       oninput="updateVideoWidth(this.value)">
               </div>
   
               <!-- Video Container -->
               <div class="video-container">
                   <video 
                       id="demo-video" 
                       autoplay 
                       muted 
                       loop 
                       playsinline 
                       style="width: 70%;">
                       <source src="src/iOS_App_demo01.mp4" type="video/mp4">
                       Your browser does not support the video tag.
                   </video>
               </div>
           </div>
       </div>
   
       <!-- JavaScript will be added here -->
   
   </body>
   </html>
   

2. Apply Styling with CSS

   Incorporate CSS to style the modal, its content, the paragraph, slider, and video to ensure proper alignment, responsiveness, and visual appeal.

   <style>
   /* Paragraph Styling */
   .demo-paragraph {
       text-align: left;
       font-size: 16px;
       line-height: 1.5;
       margin-bottom: 20px;
   }
   
   /* Modal Background */
   .modal {
       display: none; /* Initially hidden */
       position: fixed;
       z-index: 1000;
       left: 0;
       top: 0;
       width: 100%;
       height: 100%;
       overflow: auto;
       background-color: rgba(0,0,0,0.5); /* Semi-transparent background */
   }
   
   /* Modal Content Box */
   .modal-content {
       background-color: #fefefe;
       margin: 5% auto; /* Vertically and horizontally centered */
       padding: 20px;
       border: 1px solid #888;
       width: 80%;
       max-width: 800px;
       border-radius: 8px;
       position: relative;
       animation-name: animatetop;
       animation-duration: 0.4s;
       box-shadow: 0 5px 15px rgba(0,0,0,0.3);
   }
   
   /* Entrance Animation */
   @keyframes animatetop {
       from {transform: translateY(-50px); opacity: 0}
       to {transform: translateY(0); opacity: 1}
   }
   
   /* Close Button Styling */
   .close {
       color: #aaa;
       position: absolute;
       top: 10px;
       right: 20px;
       font-size: 28px;
       font-weight: bold;
       cursor: pointer;
   }
   
   .close:hover,
   .close:focus {
       color: black;
       text-decoration: none;
   }
   
   /* Prevent Background Scrolling When Modal is Open */
   body.modal-open {
       overflow: hidden;
   }
   
   /* Slider Container Styling */
   .slider-container {
       margin-bottom: 20px;
   }
   
   .slider-container label {
       display: block;
       margin-bottom: 10px;
       font-weight: bold;
   }
   
   .slider-container input[type="range"] {
       width: 100%;
   }
   
   /* Centering the Video */
   .video-container {
       display: flex;
       justify-content: center;
   }
   
   /* Responsive Video Styling */
   video {
       max-width: 100%;
       height: auto;
       border: 2px solid #ccc;
       border-radius: 10px;
   }
   </style>
   

3. Implement Functionality with JavaScript

   Add JavaScript to handle the automatic opening of the modal upon page load, closing mechanisms, and dynamic adjustment of the video width based on the slider's value.

   <script>
   // Retrieve Modal and Close Button Elements
   const modal = document.getElementById('video-modal');
   const closeBtn = document.querySelector('.close');
   
   /**
    * Opens the modal and prevents background scrolling.
    */
   function openModal() {
       modal.style.display = 'block';
       document.body.classList.add('modal-open');
   }
   
   /**
    * Closes the modal, restores background scrolling, and resets the video.
    */
   function closeModal() {
       modal.style.display = 'none';
       document.body.classList.remove('modal-open');
       const video = document.getElementById('demo-video');
       video.pause();
       video.currentTime = 0;
   }
   
   /**
    * Updates the video width based on the slider value.
    * @param {number} value - The current value of the slider.
    */
   function updateVideoWidth(value) {
       const video = document.getElementById('demo-video');
       video.style.width = value + '%';
       document.getElementById('slider-value').textContent = value + '%';
   }
   
   /**
    * Initializes the modal to open automatically when the page loads.
    */
   window.onload = function() {
       openModal();
   }
   
   /**
    * Closes the modal when the close button is clicked.
    */
   closeBtn.onclick = function() {
       closeModal();
   }
   
   /**
    * Closes the modal when a click occurs outside the modal content.
    */
   window.onclick = function(event) {
       if (event.target == modal) {
           closeModal();
       }
   }
   
   /**
    * Allows closing the modal using the 'Escape' key for accessibility.
    */
   window.addEventListener('keydown', function(event) {
       if (event.key === 'Escape' && modal.style.display === 'block') {
           closeModal();
       }
   });
   </script>
   

Modal Pop-up with Interactive Features

This document provides a detailed explanation of the implementation of a modal pop-up window on a webpage. The modal incorporates various interactive elements, including an automatically repeating video, a slider for adjusting video width, multiple methods for closing the modal, and session persistence to enhance user experience. The guide is structured to facilitate easy understanding, reproduction, and maintenance of the modal's functionalities.

1. HTML Structure

The HTML structure establishes the foundational elements of the modal, encompassing the title, description, slider, checkbox, and video components. The modal is designed to be initially hidden and becomes visible based on user interactions or predefined conditions.

<!-- Modal Structure -->
<div id="video-modal" class="modal" aria-hidden="true" role="dialog" aria-labelledby="modal-title">
    <div class="modal-content" role="document" tabindex="-1">
        <!-- Close Button -->
        <span class="close" aria-label="Close Modal">×</span>
        
        <!-- Title -->
        <h2 class="modal-title" id="modal-title">Demo Video for the Upcoming iOS App</h2>
        
        <!-- Description Paragraph -->
        <p class="modal-description">
            Currently experiencing technical issues with submission to the Apple App Store, but expected to be released in 2025, in combination with Apple Vision Pro.
        </p>

        <!-- Slider for Adjusting Video Width -->
        <div class="slider-container">
            <label for="width-slider">Adjust video width: <span id="slider-value">50%</span></label>
            <div class="slider-wrapper">
                <input 
                    type="range" 
                    id="width-slider" 
                    min="30" 
                    max="100" 
                    value="50" 
                    step="10"
                    aria-valuemin="30"
                    aria-valuemax="100"
                    aria-valuenow="50"
                    aria-label="Adjust video width"
                    oninput="updateVideoWidth(this.value)">
            </div>
        </div>

        <!-- Checkbox to Prevent Modal from Showing Again in Session -->
        <div class="checkbox-container">
            <input type="checkbox" id="dont-show-checkbox" onclick="handleCheckbox(this)">
            <label for="dont-show-checkbox">Don't show this again</label>
        </div>

        <!-- Video Container -->
        <div class="video-container">
            <video 
                id="demo-video" 
                autoplay 
                muted 
                loop 
                playsinline 
                style="width: 50%;">
                <source src="src/iOS_App_demo01.mp4" type="video/mp4">
                Your browser does not support the video tag.
            </video>
        </div>
    </div>
</div>

• Modal Container (<div id="video-modal" class="modal">): Serves as the main container for the modal, which is hidden by default.
• Modal Content (<div class="modal-content">): Encapsulates all modal elements, including the close button, title, description, slider, checkbox, and video.
• Close Button (<span class="close">×</span>): Facilitates the closure of the modal.
• Slider (<input id="width-slider">): Allows users to adjust the width of the video.
• Checkbox (<input id="dont-show-checkbox">): Enables users to opt out of seeing the modal again during the current session.
• Video (<video id="demo-video">): Plays automatically, muted, and loops continuously.

2. CSS Styling

The CSS defines the visual presentation of the modal and its components, ensuring an aesthetically pleasing and responsive design. It handles the layout, animations, responsiveness, and overall user interface enhancements.

/* Modal Background */
.modal {
    display: none; /* Initially hidden */
    position: fixed;
    z-index: 1000;
    left: 0;
    top: 0;
    width: 100%;
    height: 100%;
    overflow: auto;
    background-color: rgba(0,0,0,0.5); /* Semi-transparent background */
}

/* Modal Content Box */
.modal-content {
    background-color: #fefefe;
    margin: 5% auto; /* Vertically and horizontally centered */
    padding: 20px;
    border: 1px solid #888;
    width: 80%;
    max-width: 800px;
    border-radius: 8px;
    position: relative;
    animation-name: animatetop;
    animation-duration: 0.4s;
    box-shadow: 0 5px 15px rgba(0,0,0,0.3);
    outline: none; /* Remove default outline */
}

/* Entrance Animation */
@keyframes animatetop {
    from {transform: translateY(-50px); opacity: 0}
    to {transform: translateY(0); opacity: 1}
}

/* Close Button Styling */
.close {
    color: #aaa;
    position: absolute;
    top: 10px;
    right: 20px;
    font-size: 28px;
    font-weight: bold;
    cursor: pointer;
}

.close:hover,
.close:focus {
    color: black;
    text-decoration: none;
}

/* Prevent Background Scrolling When Modal is Open */
body.modal-open {
    overflow: hidden;
}

/* Slider Container Styling */
.slider-container {
    margin-bottom: 20px;
    display: flex;
    align-items: center;
}

.slider-container label {
    display: block;
    margin-bottom: 10px;
    font-weight: bold;
    flex: 1;
}

.slider-wrapper {
    width: 66%; /* Set slider to 2/3 of the modal width */
}

.slider-wrapper input[type="range"] {
    width: 100%;
}

/* Checkbox Container Styling */
.checkbox-container {
    margin-bottom: 20px;
    display: flex;
    justify-content: flex-end; /* Align to the right */
    align-items: center;
}

.checkbox-container input[type="checkbox"] {
    margin-right: 10px;
}

/* Title Styling */
.modal-title {
    font-size: 24px;
    font-weight: bold;
    margin-bottom: 15px;
    text-align: center;
}

/* Description Styling */
.modal-description {
    text-align: center;
    font-size: 16px;
    margin-bottom: 20px;
}

/* Centering the Video */
.video-container {
    display: flex;
    justify-content: center;
}

/* Responsive Video Styling */
video {
    max-width: 100%;
    height: auto;
    border: 2px solid #ccc;
    border-radius: 10px;
}

/* Responsive Adjustments */
@media (max-width: 600px) {
    .modal-content {
        width: 90%;
    }
    .slider-wrapper {
        width: 100%;
    }
    .checkbox-container {
        justify-content: center;
    }
}

• Modal Visibility: The .modal class ensures the modal is hidden by default and covers the entire viewport with a semi-transparent background when displayed.
• Modal Content: Centrally positioned with padding, border-radius, box-shadow, and an entrance animation (animatetop) to enhance visual appeal.
• Close Button: Positioned at the top-right corner with hover and focus effects to improve user interaction.
• Slider and Checkbox: Utilizes Flexbox for alignment and sizing, ensuring a clean and organized layout.
• Responsive Design: Media queries adjust the modal's width and layout on smaller screens to maintain usability across devices.
• Video Styling: Ensures the video maintains its aspect ratio and is responsive within its container, with added borders and rounded corners for visual distinction.

3. JavaScript Functionality

The JavaScript code orchestrates the interactive behavior of the modal, managing its visibility, user interactions, and state persistence. It ensures a seamless and intuitive user experience by handling events and dynamically updating the modal's properties.

<script>
    // Retrieve Modal and Close Button Elements
    const modal = document.getElementById('video-modal');
    const closeBtn = document.querySelector('.close');

    /**
     * Opens the modal and prevents background scrolling.
     */
    function openModal() {
        modal.style.display = 'block';
        modal.setAttribute('aria-hidden', 'false');
        document.body.classList.add('modal-open');
        // Set focus to the modal for accessibility
        modal.querySelector('.modal-content').focus();
    }

    /**
     * Closes the modal, restores background scrolling, and resets the video.
     */
    function closeModal() {
        modal.style.display = 'none';
        modal.setAttribute('aria-hidden', 'true');
        document.body.classList.remove('modal-open');
        const video = document.getElementById('demo-video');
        video.pause();
        video.currentTime = 0;
    }

    /**
     * Updates the video width based on the slider value.
     * @param {number} value - The current value of the slider.
     */
    function updateVideoWidth(value) {
        const video = document.getElementById('demo-video');
        video.style.width = value + '%';
        document.getElementById('slider-value').textContent = value + '%';
        // Update ARIA attribute for accessibility
        const slider = document.getElementById('width-slider');
        slider.setAttribute('aria-valuenow', value);
    }

    /**
     * Handles the checkbox state to prevent modal from showing again in the session.
     * If checked, also closes the modal.
     * @param {HTMLInputElement} checkbox - The checkbox element.
     */
    function handleCheckbox(checkbox) {
        if (checkbox.checked) {
            sessionStorage.setItem('hideVideoModal', 'true');
            closeModal(); // Close the modal immediately when checked
        } else {
            sessionStorage.removeItem('hideVideoModal');
        }
    }

    /**
     * Initializes the modal to open automatically when the page loads,
     * unless the user has opted not to show it.
     */
    window.addEventListener('DOMContentLoaded', () => {
        const hideModal = sessionStorage.getItem('hideVideoModal');
        if (!hideModal) {
            openModal();
        }
    });

    /**
     * Closes the modal when the close button is clicked.
     */
    closeBtn.addEventListener('click', closeModal);

    /**
     * Closes the modal when a click occurs outside the modal content.
     */
    modal.addEventListener('click', function(event) {
        if (event.target === modal) {
            closeModal();
        }
    });

    /**
     * Prevents clicks inside modal-content from closing the modal.
     */
    const modalContent = document.querySelector('.modal-content');
    modalContent.addEventListener('click', function(event) {
        event.stopPropagation();
    });

    /**
     * Allows closing the modal using the 'Escape' key for accessibility.
     */
    window.addEventListener('keydown', function(event) {
        if (event.key === 'Escape' && modal.style.display === 'block') {
            closeModal();
        }
    });

    /**
     * Implements focus trapping within the modal for accessibility.
     */
    modal.addEventListener('keydown', function(event) {
        const focusableElements = modal.querySelectorAll('a[href], button, textarea, input, select, [tabindex]:not([tabindex="-1"])');
        const firstElement = focusableElements[0];
        const lastElement = focusableElements[focusableElements.length - 1];

        if (event.key === 'Tab') {
            if (event.shiftKey) { // Shift + Tab
                if (document.activeElement === firstElement) {
                    event.preventDefault();
                    lastElement.focus();
                }
            } else { // Tab
                if (document.activeElement === lastElement) {
                    event.preventDefault();
                    firstElement.focus();
                }
            }
        }
    });
</script>

1. Modal Control:
   • openModal(): Displays the modal, prevents background scrolling, and sets focus for accessibility.
   • closeModal(): Hides the modal, restores background scrolling, and resets the video playback.
2. Slider Interaction:
   • updateVideoWidth(value): Adjusts the video's width based on the slider's value and updates the displayed percentage. It also updates the ARIA attribute to reflect the current value for assistive technologies.
3. Checkbox Handling:
   • handleCheckbox(checkbox): When the checkbox is selected, it stores a flag in sessionStorage to prevent the modal from appearing again during the session and closes the modal immediately.
4. Initialization:
   • On page load (DOMContentLoaded), the script checks sessionStorage to determine whether to display the modal.
5. Event Listeners:
   • Close Button Click: Closes the modal.
   • Outside Click: Clicking outside the modal content closes the modal.
   • Inside Click: Prevents clicks inside the modal content from propagating and closing the modal.
   • Escape Key: Pressing the 'Escape' key closes the modal.
   • Focus Trapping: Ensures that keyboard navigation (Tab key) remains within the modal for accessibility.

4. Implementation Details

This section delves into the specific implementation strategies employed to achieve the desired functionalities of the modal pop-up. Each subsection highlights the relevant code snippets and explains their roles in fulfilling the respective tasks.

4.1 Video Playback: Automatically Repeating Video

The video element is configured to play automatically upon the modal's appearance and to loop continuously, ensuring uninterrupted playback.

<!-- Video Container -->
<div class="video-container">
    <video 
        id="demo-video" 
        autoplay 
        muted 
        loop 
        playsinline 
        style="width: 50%;">
        <source src="src/iOS_App_demo01.mp4" type="video/mp4">
        Your browser does not support the video tag.
    </video>
</div>

• autoplay: Initiates video playback automatically when the modal is displayed.
• muted: Mutes the video to prevent unexpected audio playback.
• loop: Enables continuous playback by looping the video upon completion.
• playsinline: Allows the video to play inline on mobile devices, preventing it from opening in full-screen mode.

4.2 Slider Control: Adjusting Video Width

A slider is integrated to allow users to adjust the video's width in increments of 10%, within a range of 30% to 100%. The default width is set to 50%.

<!-- Slider for Adjusting Video Width -->
<div class="slider-container">
    <label for="width-slider">Adjust video width: <span id="slider-value">50%</span></label>
    <div class="slider-wrapper">
        <input 
            type="range" 
            id="width-slider" 
            min="30" 
            max="100" 
            value="50" 
            step="10"
            aria-valuemin="30"
            aria-valuemax="100"
            aria-valuenow="50"
            aria-label="Adjust video width"
            oninput="updateVideoWidth(this.value)">
    </div>
</div>

/**
 * Updates the video width based on the slider value.
 * @param {number} value - The current value of the slider.
 */
function updateVideoWidth(value) {
    const video = document.getElementById('demo-video');
    video.style.width = value + '%';
    document.getElementById('slider-value').textContent = value + '%';
    // Update ARIA attribute for accessibility
    const slider = document.getElementById('width-slider');
    slider.setAttribute('aria-valuenow', value);
}

• Slider Configuration:
  • min="30" and max="100": Define the permissible range of video widths.
  • step="10": Sets the slider to increment by 10% with each movement.
  • value="50": Establishes the default video width at 50%.
• Slider Interaction (oninput Event):
  • updateVideoWidth(this.value): Invokes the updateVideoWidth function, passing the current slider value to adjust the video's width dynamically.
• updateVideoWidth Function:
  • Modifies the video's width based on the slider's value.
  • Updates the displayed percentage next to the slider label.
  • Adjusts the ARIA attribute to reflect the current value, enhancing accessibility for assistive technologies.

4.3 Multiple Close Methods

The modal incorporates three distinct methods for closure: clicking the "X" button, clicking outside the modal content area, and selecting the provided checkbox.

<!-- Close Button -->
<span class="close" aria-label="Close Modal">×</span>

/**
 * Closes the modal when the close button is clicked.
 */
closeBtn.addEventListener('click', closeModal);

/**
 * Closes the modal when a click occurs outside the modal content.
 */
modal.addEventListener('click', function(event) {
    if (event.target === modal) {
        closeModal();
    }
});

/**
 * Handles the checkbox state to prevent modal from showing again in the session.
 * If checked, also closes the modal.
 * @param {HTMLInputElement} checkbox - The checkbox element.
 */
function handleCheckbox(checkbox) {
    if (checkbox.checked) {
        sessionStorage.setItem('hideVideoModal', 'true');
        closeModal(); // Close the modal immediately when checked
    } else {
        sessionStorage.removeItem('hideVideoModal');
    }
}

1. Closing via "X" Button:
   • The close button (<span class="close">×</span>) is equipped with an event listener that triggers the closeModal function upon being clicked.
2. Closing by Clicking Outside Modal Content:
   • An event listener on the modal container detects clicks outside the modal content area. If such a click is detected (i.e., the event target is the modal itself), the closeModal function is invoked.
3. Closing via Checkbox Selection:
   • The checkbox (<input id="dont-show-checkbox">) is linked to the handleCheckbox function through the onclick attribute.
   • When the checkbox is selected, the modal is closed immediately by calling closeModal, and a flag is set in sessionStorage to prevent the modal from reappearing during the current session.

4.4 Session Persistence: Preventing Modal Reappearance

To enhance user experience, the modal incorporates a checkbox that, when selected, prevents the modal from appearing again during the current browsing session.

<!-- Checkbox to Prevent Modal from Showing Again in Session -->
<div class="checkbox-container">
    <input type="checkbox" id="dont-show-checkbox" onclick="handleCheckbox(this)">
    <label for="dont-show-checkbox">Don't show this again</label>
</div>

/**
 * Handles the checkbox state to prevent modal from showing again in the session.
 * If checked, also closes the modal.
 * @param {HTMLInputElement} checkbox - The checkbox element.
 */
function handleCheckbox(checkbox) {
    if (checkbox.checked) {
        sessionStorage.setItem('hideVideoModal', 'true');
        closeModal(); // Close the modal immediately when checked
    } else {
        sessionStorage.removeItem('hideVideoModal');
    }
}

/**
 * Initializes the modal to open automatically when the page loads,
 * unless the user has opted not to show it.
 */
window.addEventListener('DOMContentLoaded', () => {
    const hideModal = sessionStorage.getItem('hideVideoModal');
    if (!hideModal) {
        openModal();
    }
});

• Checkbox Interaction:
  • When the checkbox is clicked, the handleCheckbox function assesses its state.
  • If Checked:
    • Sets a flag (hideVideoModal) in sessionStorage to indicate the user's preference to hide the modal.
    • Invokes closeModal to immediately close the modal.
  • If Unchecked:
    • Removes the flag from sessionStorage, allowing the modal to appear in future sessions.
• Modal Initialization:
  • Upon the page's DOMContentLoaded event, the script checks sessionStorage for the hideVideoModal flag.
  • If the flag is absent, the openModal function is called to display the modal automatically.

Embedded Video

Embedding video content on webpages can be achieved using the native HTML5 <video> element for self-hosted files or by using embedded iframes for platforms like YouTube. This guide provides a comprehensive overview of both approaches, covering responsive design, autoplay policies, performance optimizations, accessibility, and solutions for dealing with YouTube embed restrictions. The aim is to offer ready-to-use templates and best practices for each scenario.

1. HTML5 <video> Embedding

   The HTML5 <video> element allows you to embed video files directly into a webpage without relying on external plugins. It supports multiple source formats and provides built-in playback controls, making it the standard choice for self-hosted videos. Below is a basic example of embedding a video with two source formats and a fallback message:

   <video controls width="640" height="360" poster="placeholder.jpg">
     <source src="video.mp4" type="video/mp4">
     <source src="video.webm" type="video/webm">
     Your browser does not support the video tag.
   </video>
   

   Explanation: The <video> element above includes controls so that the browser’s native play/pause and volume UI is shown. The width and height attributes set the display size in pixels (640×360 in this case). A poster image is specified, which will be shown as a placeholder before the video starts. Inside the <video> element, multiple <source> elements are provided: the browser will use the first source with a supported format (here, MP4 then WebM). The text “Your browser does not support the video tag.” will display only if the browser cannot play any provided format or doesn’t support HTML5 video at all.

   In practice, you should include at least an MP4 source (using H.264/AAC encoding) for maximum compatibility, since all modern browsers support MP4. Optionally, providing a WebM source (VP8/VP9) can offer better compression in browsers that support it. Older formats like Ogg Theora can be included for legacy support, but they are largely unnecessary today. The width and height attributes are also optional; you can omit them and use CSS to control sizing (which is preferable for responsive layouts, discussed later).

   Common <video> attributes:

   • controls: Displays the browser’s default video controls (play/pause, seek bar, volume, etc.). Without this, no controls are shown unless you implement custom ones via JavaScript.
   • autoplay: Starts playing the video automatically upon page load. Note that most browsers will only autoplay if the video is muted (see Autoplay and Browser Restrictions section).
   • muted: Mutes the audio by default. This is required for autoplay to work in modern browsers and is useful for background videos where sound isn’t desired.
   • loop: Causes the video to restart from the beginning after reaching the end, playing in a continuous loop.
   • poster: URL of an image to show before the video plays (or while downloading). Helps provide a preview frame instead of a black or blank box.
   • preload: Suggests how much of the file to preload. Values can be "auto" (let the browser decide or preload the whole video), "metadata" (preload only video metadata like duration), or "none" (do not preload anything until user hits play). Proper use can improve performance.
   • playsinline: Especially for mobile devices, this attribute allows the video to play inline within the page. On iPhones, for example, a video without playsinline might automatically enter fullscreen playback.
   • controlsList: A non-standard attribute (supported in some browsers like Chrome) to control which controls are shown. For instance, controlsList="nodownload" can hide the download option from the context menu.
   • crossorigin: If the video file is hosted on a different domain and you need to manipulate it with canvas or JavaScript (for example, to apply filters or analytics), set crossorigin="anonymous" and ensure the server provides appropriate CORS headers. This avoids cross-domain security issues.

   Using the native <video> element has the advantage of being plugin-free and under your control. You can style it with CSS (to an extent) and even build custom controls using JavaScript by interacting with the video’s API (e.g., play(), pause(), etc.). However, creating fully custom video players can be complex, so for basic usage the default controls are recommended. In older web development (before HTML5), videos were often embedded using Flash or other plugins via <object>/<embed> tags – those methods are now obsolete in favor of the HTML5 approach demonstrated above.

2. YouTube <iframe> Embedding

   Instead of self-hosting video files, you can embed videos from platforms like YouTube using an <iframe>. This is often convenient because YouTube handles video encoding, player UI, and bandwidth. The basic approach is to use YouTube’s embed URL within an iframe. Here is a typical example of embedding a YouTube video:

   <iframe
     width="560"
     height="315"
     src="https://www.youtube.com/embed/VIDEO_ID?playsinline=1"
     title="YouTube video player"
     frameborder="0"
     allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
     allowfullscreen
   ></iframe>
   

   Explanation: Replace VIDEO_ID in the src URL with the unique identifier of the YouTube video you want to embed (this is the part after v= in a normal YouTube link). In the example above, width and height define the embedded player’s size in pixels (560×315 is a common default for a 16:9 video). The title attribute provides an accessible label for the iframe (important for screen readers). The frameborder="0" attribute removes the old-style border around the iframe (though in HTML5 this can also be achieved with CSS). The allowfullscreen attribute permits the video to be viewed in fullscreen mode when the user clicks the fullscreen button.

   The allow attribute in the iframe is set with a list of allowed features: this example includes autoplay, encrypted-media, clipboard-write, accelerometer, gyroscope, and picture-in-picture. These enable the corresponding functionalities inside the iframe (for instance, allowing the video to autoplay if otherwise permitted, enabling the clipboard copy feature on YouTube’s UI, or allowing the video to go into Picture-in-Picture mode on supporting browsers). The ?playsinline=1 query parameter we added in the URL is a hint for iOS devices to play the video inline within the page instead of forcing fullscreen playback.

   YouTube iframes can be further customized via URL parameters. Here are some common YouTube embed query parameters and their effects:

   URL Parameter	Effect
   autoplay=1	Start playing the video automatically. (Note: The video will usually need to be muted or user-interaction will be required due to browser autoplay policies.)
   mute=1	Start the video with sound muted. This is often used in conjunction with autoplay to allow the video to play without user interaction.
   controls=0	Hide the YouTube player controls (play/pause buttons, etc.). Use this if you want to provide your own controls or have a non-interactive video display. (Default is 1, which shows controls.)
   rel=0	When the video finishes, show related videos from the same channel only, rather than random suggestions. Prior to 2018, rel=0 would hide all related videos, but now it’s limited to the same channel.
   loop=1	Loop the video. If using this for a single video, you must also add playlist=[VIDEO_ID] (the same video’s ID) in the URL to make looping work.
   playlist=VIDEO_IDs	A comma-separated list of video IDs to play in sequence. Often used along with loop=1 to loop a single video or to create a playlist of multiple videos in one embed.
   modestbranding=1	Removes the YouTube logo from the control bar (a small YouTube text will still appear, but the big logo watermark is hidden).
   start=30	Start playback at 30 seconds into the video (replace 30 with any number of seconds).
   end=60	Stop playback at 60 seconds (the video will stop at that point instead of playing to the end).
   cc_load_policy=1	Show closed captions (subtitles) by default if the video has them.
   cc_lang_pref=en	Set the default captions language to English (using ISO language code, here "en"). You can change the code for other languages as needed.
   playsinline=1	Allows inline playback on mobile devices (particularly iOS). We included this in the example iframe URL to avoid fullscreen enforcement on iPhones.
   enablejsapi=1	Enables the JavaScript API. Required if you plan to control the player via JavaScript (using YouTube IFrame Player API). When using this, you should also specify an origin parameter (your domain) for security.

   By combining these parameters in the src URL, you can tailor the embed to your needs. For example, to embed a video that autoplays muted and loops continuously without showing controls or related videos, your src could look like: https://www.youtube.com/embed/VIDEO_ID?autoplay=1&mute=1&loop=1&playlist=VIDEO_ID&controls=0&rel=0. It’s generally best to only use the parameters you need, as too many can complicate the URL.

   Privacy Consideration: If you need to minimize tracking, YouTube offers a privacy-enhanced embed mode. This is achieved by using the youtube-nocookie.com domain for the embed. For instance: src="https://www.youtube-nocookie.com/embed/VIDEO_ID". In this mode, YouTube will not set cookies until the user interacts with the video (though their IP may still be transmitted to load the video). This can help with privacy compliance, but functionally the video will appear and play the same way.

   One thing to be aware of is that some YouTube videos may not allow embedding on external sites. If the video’s owner has disabled embedding, the player will display a message like “Video unavailable” or prompt the user to watch it on YouTube. Similarly, age-restricted content might not play in an iframe without user login. In the next sections, we will explore how to handle responsive design for video embeds, as well as strategies for autoplay, performance, accessibility, and dealing with such restrictions.

Responsiveness Techniques for Embedded Videos

By default, an embedded video (whether an HTML5 video element or a YouTube iframe) with fixed width and height will not automatically resize on smaller screens. To ensure videos look good on all devices (desktop, tablet, mobile), you need to make the embed responsive – i.e., able to scale proportionally to the width of its container. There are two primary approaches to achieve responsive videos:

1. Using a Wrapper and Percentage Padding (Classic Method)

   This technique wraps the video element (or iframe) in a container div, which uses CSS to enforce a specific aspect ratio by using percentage padding. It has been a long-standing solution and works across all browsers. The trick is to give the container a bottom padding that is a percentage of its width (which defines the aspect ratio), then absolutely position the video inside it. For example, for a 16:9 aspect ratio video, use 56.25% padding (since 9/16 = 0.5625).

   <!-- HTML -->
   <div class="video-container">
     <iframe src="https://www.youtube.com/embed/VIDEO_ID" frameborder="0" allowfullscreen></iframe>
   </div>
   
   <!-- CSS -->
   <style>
     .video-container {
       position: relative;
       width: 100%;
       height: 0;
       padding-bottom: 56.25%; /* 16:9 aspect ratio */
     }
     .video-container iframe,
     .video-container video {
       position: absolute;
       top: 0;
       left: 0;
       width: 100%;
       height: 100%;
     }
   </style>
   

   Explanation: The .video-container div is set to position: relative; and given width: 100%; (so it will scale to its parent’s width) but a fixed height: 0. The crucial part is padding-bottom: 56.25%;, which creates an internal space that is 56.25% of the container’s width, effectively enforcing a 16:9 ratio (for a different aspect ratio, adjust this percentage accordingly: e.g., 75% for 4:3 videos). The video iframe (or a <video> tag, which we also targeted in the CSS) is then absolutely positioned to fill this container (100% width and height, with top:0 and left:0). This way, as the container’s width changes (for example, on a smaller screen), its height adjusts to maintain the aspect ratio, and the video inside scales up or down to fit.

   You can use this method for multiple videos by reusing the video-container class for each video. It ensures your embed is fluid. Note that in this approach, the width of the video will never exceed the container’s width (often the container might be constrained by a parent element’s max-width). If you want to limit how large the video can get, you can add a max-width to the .video-container (for instance, max-width: 800px; to not grow beyond 800px even on big screens). The aspect ratio will still hold at smaller sizes down to very narrow widths.

2. Using CSS aspect-ratio (Modern Method)

   Newer CSS allows specifying an aspect ratio directly, eliminating the need for a wrapper div. This method is cleaner since you can apply it to the <iframe> or <video> element itself. All modern browsers support aspect-ratio (though Internet Explorer does not, which is generally no longer a concern). To use this, set the element to width 100% and define its aspect ratio in CSS:

   /* CSS */
   .responsive-video {
     width: 100%;
     aspect-ratio: 16/9;
     /* Optional: ensure it doesn't overflow its container */
     max-width: 100%;
   }
   

   In your HTML, give the video iframe (or <video>) a class of responsive-video (or you can target it via other selectors) and don’t hardcode the height attribute. The CSS above will make the element take the full width of its container and automatically calculate the height using a 16:9 ratio. The max-width: 100% ensures it doesn’t overflow its container (which can be useful if the container has padding or other styling). If you need a different ratio, simply change 16/9 to your desired width/height ratio (for example 4/3 for 4:3).

   This modern method achieves the same result as the padding hack but with less markup and more clarity. However, if you need to support very old browsers, you might still fall back to the wrapper technique. Otherwise, aspect-ratio is the recommended approach for simplicity.

   In summary, a responsive video embed can be implemented either by using a containing element to enforce aspect ratio (compatible with essentially all browsers), or by leveraging the aspect-ratio CSS property for a cleaner solution in modern environments. Both methods ensure that your videos resize gracefully on different screen sizes without distortion.

Written on April 26, 2025

Embedding and Optimizing HTML5 Video

This document provides an integrated and refined overview of various methods and best practices for embedding video content using the HTML5 <video> element. It consolidates multiple approaches, ensuring that all discussed features, attributes, stylistic considerations, and optimizations are included. This guide maintains a formal tone and presents information in a hierarchical, structured manner. All examples prefer inline styling to avoid interaction with external styling or other HTML files. The provided instructions, thoughts, and ideas are arranged cohesively to facilitate informed decision-making after reading through the material.

1. Basic HTML Structure and Attributes

Embedding a video with the HTML5 <video> element involves specifying one or more <source> elements and providing fallbacks. At its core, the simplest example includes only the <video> tag, a single source, and a fallback message:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Basic Video Embed</title>
</head>
<body>
  <video style="width:100px; height:auto;" autoplay loop muted>
    <source src="src/Sora_Medusa20241219.mp4" type="video/mp4">
    Your browser does not support the video tag.
  </video>
</body>
</html>

• width and height (inline style): Ensures control over the video’s dimensions. A fixed width (e.g., 100px) or a responsive approach (width:100%; height:auto;) can be used.
• autoplay: Initiates playback as soon as the video is able to do so without interruption.
• loop: Replays the video automatically after it concludes.
• muted: Ensures browsers allow autoplay without user interaction, as most browsers block autoplay with sound.
• playsinline: Allows the video to play inline on mobile devices rather than switching to fullscreen.
• controls: Displays playback controls, enabling manual interactions such as play, pause, and volume adjustments.
• preload: Determines what data is loaded when the page initially loads. Common values include none, metadata, and auto. For performance, using metadata or none may be preferable.
• type="video/mp4": Specifies the MIME type of the source, aiding browser compatibility and handling.

2. Responsive Embedding and Layout Considerations

To ensure the video adapts to different screen sizes, inline styles can be used to achieve a responsive layout. Applying width: 100%; height: auto; allows the video to scale according to its container:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Responsive Video Embed</title>
</head>
<body>
  <video style="width:100%; height:auto;" autoplay loop muted playsinline>
    <source src="src/Sora_Medusa20241219.mp4" type="video/mp4">
    Your browser does not support the video tag.
  </video>
</body>
</html>

To avoid external CSS interactions, inline styles can be maintained. For further refinement, a containing element (e.g., a <div> or <figure>) with its own max-width can limit the video’s maximum size while preserving responsiveness. For example:

<figure style="width:100%; max-width:800px; margin:0 auto;">
  <video style="width:100%; height:auto;" autoplay loop muted playsinline>
    <source src="src/Sora_Medusa20241219.mp4" type="video/mp4">
    Your browser does not support the video tag.
  </video>
</figure>

3. Multiple Video Formats for Wider Browser Support

To ensure compatibility across a range of browsers, multiple source formats can be included. Browsers will choose the first format they can play:

<video style="width:100%; height:auto;" autoplay loop muted playsinline>
  <source src="src/Sora_Medusa20241219.mp4" type="video/mp4">
  <source src="src/Sora_Medusa20241219.webm" type="video/webm">
  <source src="src/Sora_Medusa20241219.ogv" type="video/ogg">
  Your browser does not support the video tag.
</video>

4. Accessibility and Semantics

Semantically grouping a video with its caption enhances accessibility and SEO. The <figure> and <figcaption> elements are often used together. This approach allows the caption to describe the video’s content meaningfully. An italicized <em> can distinguish the text stylistically:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Embed Sora Video</title>
</head>
<body>

  <h1>My Sora Video</h1>

  <figure>
    <video style="width:100%; height:auto;" controls loop muted playsinline preload="metadata">
      <source src="src/Sora_Medusa20241219.mp4" type="video/mp4">
      Your browser does not support the video tag.
    </video>
    <figcaption>
      <em>Medusa with serpentine hair inside a Greek temple, embodying Greek mythology as she awaits her encounter with Perseus. Created using Sora OpenAI.</em>
    </figcaption>
  </figure>

</body>
</html>

Caption Refinement Considerations:

• The caption can mention creative tools (e.g., Sora OpenAI) and provide a descriptive narrative.
• Additional ARIA attributes or <track> elements with captions can improve accessibility further. For example:

<video style="width:100%; height:auto;" controls loop muted playsinline preload="metadata">
  <source src="src/Sora_Medusa20241219.mp4" type="video/mp4">
  <track kind="captions" src="src/captions.vtt" srclang="en" label="English">
  Your browser does not support the video tag.
</video>

This ensures users with hearing impairments can access the content through captions.

5. Autoplay Behavior and Browser Policies

Modern browsers often block videos from autoplaying if they contain sound. Using the muted attribute is recommended to ensure autoplay functionality. If desired, the autoplay attribute can be added to start playback immediately, as long as the video remains muted. For user-driven control, consider removing autoplay and relying solely on controls.

6. Performance Optimization Techniques

Efficient loading of videos enhances user experience, especially on pages with multiple videos or limited bandwidth environments. Consider these methods:

Preload Settings:

• preload="metadata" loads only essential information (such as duration and dimensions) without downloading the entire file.
• preload="none" further reduces initial load by not loading any video data until the user initiates playback or the video appears in the viewport.

Video Compression:

Compressing the video file before embedding reduces file size and improves load times. Tools such as HandBrake can help optimize the video without significantly sacrificing quality.

Lazy Loading with JavaScript:

For scenarios involving multiple videos, lazy loading prevents videos from loading until they are near or within the user’s viewport. A JavaScript Intersection Observer can detect when a video enters the viewport and then set the src dynamically:

<video 
  style="width:100%; height:auto;" 
  controls loop muted playsinline
  preload="none" 
  data-src="src/Sora_Medusa20241219.mp4"
  class="lazy-video"
>
  Your browser does not support the video tag.
</video>

<script>
  document.addEventListener("DOMContentLoaded", function() {
    const lazyVideos = document.querySelectorAll('video.lazy-video');

    if ("IntersectionObserver" in window) {
      let lazyVideoObserver = new IntersectionObserver(function(entries, observer) {
        entries.forEach(function(entry) {
          if (entry.isIntersecting) {
            let video = entry.target;
            video.src = video.dataset.src;
            video.load();
            video.classList.remove("lazy-video");
            lazyVideoObserver.unobserve(video);
          }
        });
      });

      lazyVideos.forEach(function(video) {
        lazyVideoObserver.observe(video);
      });
    } else {
      // Fallback for browsers without IntersectionObserver
      lazyVideos.forEach(function(video) {
        video.src = video.dataset.src;
        video.load();
      });
    }
  });
</script>

This approach ensures minimal initial page load overhead and only loads videos as needed.

Written on December 20th, 2024

How to Embed a Youtube Video and Ensure Visibility on External Website

This document presents a comprehensive and structured reference for two primary tasks:

1. Embedding a YouTube video within a webpage using <iframe> code.
2. Enabling embedding in the YouTube Studio so that the video can be viewed directly on external websites.

The content is divided into two main sections—Part A (Embedding Essentials) and Part B (YouTube Content Settings)—offering a clear hierarchy to help developers and content owners seamlessly integrate YouTube videos into their websites.

Part A: Embedding Essentials

When embedding a YouTube video, substituting a local <video> tag with an <iframe> tag is typically the best practice. This approach ensures:

• Seamless user experience
• Bandwidth offloading to YouTube’s servers
• Standard playback controls readily available

1. Basic Embed Code

   <iframe 
     src="https://www.youtube.com/embed/[VIDEO_ID]"
     title="Descriptive Title"
     frameborder="0"
     allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
     allowfullscreen
   ></iframe>
   

   • src: Accepts the embed URL in the format https://www.youtube.com/embed/VIDEO_ID.
   • title: Serves as an accessibility aid and describes the video content.
   • frameborder: Commonly set to 0 for a borderless frame.
   • allow: Grants permission for autoplay, clipboard access, and other features.
   • allowfullscreen: Enables full-screen functionality on supported devices.

2. Responsive Design Tips

   Maintaining a 16:9 aspect ratio for modern video content is a common practice. Employing a CSS trick with padding-bottom ensures responsiveness across various screen sizes:

   <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;">
     <iframe 
       src="https://www.youtube.com/embed/[VIDEO_ID]" 
       style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"
       frameborder="0"
       allowfullscreen
     ></iframe>
   </div>
   

   • Container Setup
     • position: relative; padding-bottom: 56.25%;: Helps maintain the 16:9 ratio (56.25% = 9/16 × 100).
     • height: 0; overflow: hidden;: Ensures the container scales properly.
   • Absolute Positioning
     • position: absolute; top: 0; left: 0;: Anchors the <iframe> to the top-left corner.
     • width: 100%; height: 100%;: Stretches the <iframe> to match the container dimensions.

3. Actual Example

   Below is a practical example that demonstrates embedding a specific YouTube video (gqth-OGlzS0) within a <figure> element, including a caption:

   <figure>
     <iframe
       style="width: 100%; height: 500px;"
       src="https://www.youtube.com/embed/gqth-OGlzS0"
       title="Medusa with serpentine hair inside a Greek temple"
       frameborder="0"
       allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
       allowfullscreen
     ></iframe>
     <figcaption>
       <em>Medusa with serpentine hair inside a Greek temple, embodying Greek mythology as she awaits her encounter with Perseus. Created using Sora OpenAI.</em>
     </figcaption>
   </figure>
   

   This example uses inline styling to establish a fixed width (100%) and height (500px). The <figcaption> provides descriptive information about the video content.

Part B: YouTube Content Settings for Allowing Embedding

Enabling Embedding in YouTube Studio

By default, some videos might restrict embedding, leading to the message:

“Playback on other websites has been disabled by the video owner”

To rectify this, Allow embedding must be enabled in the video’s settings.

┌────────────────────────────┐
│   Sign in to YouTube       │
└───────────────┬────────────┘
                ↓
┌──────────────────────────────────┐
│   Access YouTube Studio         │
└───────────────┬─────────────────┘
                ↓
┌──────────────────────────────────┐
│   Open Content (Videos)         │
└───────────────┬─────────────────┘
                ↓
┌──────────────────────────────────┐
│   Select the Desired Video       │
│   (Edit Details)                 │
└───────────────┬─────────────────┘
                ↓
┌──────────────────────────────────┐
│   Expand "More options"          │
│   Check "Allow embedding"        │
└───────────────┬─────────────────┘
                ↓
┌──────────────────────────────────┐
│   Save Changes                   │
└──────────────────────────────────┘

Additional Notes

1. Age-Restricted Content: May be subject to embedding limitations due to policy.
2. Copyright Restrictions: Copyright holders may disable embedding.
3. Content Security Policy (CSP): Websites must allow www.youtube.com and youtube.com in their CSP to embed videos.

Written on December 25th, 2024

Responsive YouTube Embedding

Ensuring that embedded YouTube videos remain responsive and maintain their aspect ratio across various devices is a crucial aspect of modern web design. The padding-bottom hack is a widely adopted method for achieving a responsive video embed that preserves the video’s original aspect ratio. This guide provides a comprehensive explanation of this technique, its underlying principles, and a systematic approach to apply it to similar tasks. The discussion also integrates comparative insights with alternative methods and offers structured guidance for future reference.

The Padding-Bottom Technique

The padding-bottom method utilizes a container <div> element with CSS styling that maintains a consistent aspect ratio for the embedded <iframe>. The following code snippet demonstrates the technique:

<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;">
  <iframe 
    src="https://www.youtube.com/embed/M-8FksMVAdU?autoplay=0" 
    style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;" 
    frameborder="0" 
    allow="accelerometer; encrypted-media; gyroscope; picture-in-picture" 
    allowfullscreen>
  </iframe>
</div>

• Container <div> Element:
  • position: relative; Establishes a relative positioning context for the inner <iframe> element.
  • padding-bottom: 56.25%; Derived from the 16:9 aspect ratio, this ensures that the container’s height scales proportionally to its width. Calculation: 9 ÷ 16 = 0.5625 or 56.25%.
  • height: 0; Combined with padding-bottom, this setting allows the container to size itself according to the width.
  • overflow: hidden; Prevents content spillover, ensuring a clean embed without scrollbars.
• Embedded <iframe> Element:
  • position: absolute; top: 0; left: 0; Positions the <iframe> at the top-left corner of the container.
  • width: 100%; height: 100%; Forces the <iframe> to fill the entire container space, maintaining the 16:9 aspect ratio.
  • frameborder="0"; allowfullscreen; Enhances the viewing experience by removing borders and enabling full-screen mode.
  • src attribute includes ?autoplay=0, ensuring that autoplay is disabled.

Rationale Behind the 56.25% Padding-Bottom

The value 56.25% is critical in maintaining a 16:9 aspect ratio, the standard for most YouTube videos:

• Aspect Ratio Calculation: A 16:9 ratio implies that for every 16 units of width, there are 9 units of height.
• Conversion to Percentage: The height-to-width ratio is 9/16 = 0.5625. Multiplying by 100 converts this to a percentage: 56.25%.
• Application: By applying padding-bottom: 56.25%, the container automatically scales its height based on its width, preserving the 16:9 ratio irrespective of screen size.

Comparative Options and Considerations

While the padding-bottom hack is preferred for its broad compatibility and reliability, modern CSS offers an alternative using the aspect-ratio property. However, the padding-bottom method is favored for its proven effectiveness and minimal reliance on browser support for newer CSS features.

Applying the Technique to Future Tasks

This responsive embedding method can be adapted for other videos or iframes that require a fixed aspect ratio. The following steps summarize the approach:

1. Determine the Desired Aspect Ratio: Calculate the appropriate padding-bottom percentage using the formula:

   padding-bottom = (height / width) * 100%

   For a 4:3 video, for example, this would be (3 ÷ 4) * 100% = 75%.
2. Structure the Container: Use a <div> with the calculated padding-bottom, position: relative;, height: 0;, and overflow: hidden;.
3. Embed the <iframe>: Within the container, position the <iframe> absolutely at top-left, set its width and height to 100%, and include necessary attributes such as frameborder, allowfullscreen, and source modifications like ?autoplay=0.

Written on January 14, 2025

Responsive YouTube Video Embed

This code snippet demonstrates how to embed a YouTube video in a responsive manner using inline CSS. In this example, an <iframe> element is used along with inline styling to ensure that the video:

• Fills the available width by setting width: 100%;.
• Maintains the correct aspect ratio by using aspect-ratio: 16 / 9;, which preserves the typical 16:9 video format.
• Displays without a border by setting border: 0;.

<iframe src="https://www.youtube.com/embed/bzQd86tBzek" 
        title="YouTube video player" 
        allowfullscreen 
        style="width: 100%; aspect-ratio: 16 / 9; border: 0;"></iframe>

This approach makes it easy to embed videos that automatically adapt to different screen sizes while retaining the proper display format without additional external CSS files.

<iframe
  src="https://www.youtube.com/embed/4HCv9tabFQ8?start=137"
  title="~~~~~~~~~"
  style="width: 100%; aspect-ratio: 16/9; border: 0;"
  allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
  allowfullscreen>
</iframe>

Written on April 16, 2025

How to Circumvent YouTube Embed Issues (Written April 16, 2025)

This document serves as a reference to recall the methods employed to address the situation in which a YouTube video displays "This video is unavailable" within an iframe, despite the direct link functioning properly in a browser. The strategies described below are intended to offer a range of approaches that may be implemented when encountering such restrictions.

1. Clickable Thumbnail Overlay

   This approach displays a thumbnail image (typically using YouTube’s thumbnail URL) with an overlaid play button. Upon activation, the thumbnail may either redirect the visitor to the YouTube video page or dynamically replace the thumbnail with the embedded iframe.

   Option A: Redirect to YouTube

   <div id="video-container" style="position: relative; width: 100%; padding-bottom: 56.25%; cursor: pointer;">
     <!-- Use YouTube's thumbnail URL format -->
     <img src="https://img.youtube.com/vi/JoinXlIH8uo/hqdefault.jpg" alt="Video Thumbnail" style="position: absolute; width: 100%; height: 100%; object-fit: cover;">
     <!-- Optional overlay with a play icon -->
     <div style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; display: flex; align-items: center; justify-content: center;">
       <img src="play-icon.png" alt="Play" style="width: 60px; height: 60px;">
     </div>
   </div>
   <script>
     document.getElementById("video-container").addEventListener("click", function() {
       // Redirect to the full YouTube video page
       window.location.href = "https://www.youtube.com/watch?v=JoinXlIH8uo&list=WL&index=19&t=24s";
     });
   </script>    

   Option B: Load the Iframe Dynamically

   <div id="video-container" style="position: relative; width: 100%; padding-bottom: 56.25%; cursor: pointer;">
     <img src="https://img.youtube.com/vi/JoinXlIH8uo/hqdefault.jpg" alt="Video Thumbnail" style="position: absolute; width: 100%; height: 100%; object-fit: cover;">
     <div style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; display: flex; align-items: center; justify-content: center;">
       <img src="play-icon.png" alt="Play" style="width: 60px; height: 60px;">
     </div>
   </div>
   <script>
     document.getElementById("video-container").addEventListener("click", function() {
       // Replace the thumbnail with the embed code
       this.innerHTML = '<iframe src="https://www.youtube.com/embed/JoinXlIH8uo?list=WL&index=19&start=24" ' +
                        'title="YouTube video player" allowfullscreen ' +
                        'style="position: absolute; width: 100%; height: 100%; border: 0;"></iframe>';
     });
   </script>    

2. Modal or Lightbox Display

   This method involves using a modal or lightbox to display the video. A hidden modal is prepared in advance, and when activated by a click on the thumbnail, the modal becomes visible with the embedded video. This approach maintains the visitor on the site and defers the loading of the iframe until after the interaction.

   <!-- Modal Container (initially hidden) -->
   <div id="videoModal" style="display: none; position: fixed; z-index: 1000; left: 0; top: 0; width: 100%; height: 100%; overflow: auto; background-color: rgba(0, 0, 0, 0.8);">
     <div style="margin: 10% auto; width: 80%; max-width: 800px; position: relative;">
       <!-- Close button -->
       <span id="closeModal" style="position: absolute; top: 10px; right: 20px; color: #fff; font-size: 30px; cursor: pointer;">×</span>
       <iframe src="" id="modalVideo" title="YouTube video player" allowfullscreen style="width: 100%; aspect-ratio: 16 / 9; border: 0;"></iframe>
     </div>
   </div>
   
   <!-- Thumbnail / Trigger -->
   <div id="openModal" style="position: relative; cursor: pointer; width: 100%; max-width: 800px; margin: auto;">
     <img src="https://img.youtube.com/vi/JoinXlIH8uo/hqdefault.jpg" alt="Video Thumbnail" style="width: 100%; display: block;">
     <div style="position: absolute; top: 50%; left: 50%; transform: translate(-50%, -50%);">
       <img src="play-icon.png" alt="Play" style="width: 60px; height: 60px;">
     </div>
   </div>
   
   <script>
     // Open modal and load video when the thumbnail is clicked
     document.getElementById("openModal").addEventListener("click", function() {
       document.getElementById("modalVideo").src = "https://www.youtube.com/embed/JoinXlIH8uo?list=WL&index=19&start=24";
       document.getElementById("videoModal").style.display = "block";
     });
   
     // Close modal when the close icon is clicked
     document.getElementById("closeModal").addEventListener("click", function() {
       document.getElementById("videoModal").style.display = "none";
       // Clear the src to stop the video when the modal is closed
       document.getElementById("modalVideo").src = "";
     });
   </script>    

3. User-Initiated Iframe Loading (On-Click Button)

   This strategy involves not preloading the iframe at all. Instead, a placeholder container and a "Play Video" button are provided. The embedded iframe is only loaded when the button is activated.

   <div id="videoContainer" style="width: 100%; aspect-ratio: 16 / 9; border: 1px solid #ccc;"></div>
   <button id="loadVideo" style="margin-top: 10px;">Play Video</button>
   
   <script>
     document.getElementById("loadVideo").addEventListener("click", function() {
       // Insert the iframe into the container when the button is clicked
       var container = document.getElementById("videoContainer");
       container.innerHTML = '<iframe src="https://www.youtube.com/embed/JoinXlIH8uo?list=WL&index=19&start=24" ' +
                             'title="YouTube video player" allowfullscreen style="width:100%; height:100%; border:0;"></iframe>';
       // Optionally hide the button after loading the video
       this.style.display = "none";
     });
   </script>    

Written on April 16, 2025

Toggle

Reliable Methods for Storing Dev Mode Status in Web Applications

In web application development, maintaining consistent state across sessions and page loads is crucial. Inconsistencies in storing the "dev mode" status can lead to unpredictable behavior and hinder the development process. This document explores various reliable methods for storing the dev mode status, providing detailed sample code for each approach to aid in implementation.

Implementing a Toggle Switch with Dev-Mode Functionality

This guide provides a refined approach to integrating a toggle switch for toggling developer-specific content and managing dynamic UI elements. Key sections include HTML, updated CSS styles, and simplified JavaScript that ensures the toggle state persists across browser sessions.

1. Add the Toggle Switch to the Page

   Replace the existing checkbox with the following HTML structure to create a toggle switch:

   <label class="switch">
     <input type="checkbox" id="devToggle">
     <span class="slider"></span>
   </label>
   <span style="margin-left: 10px;">Dev Mode</span>
   

   This transforms the checkbox into a modern toggle switch.

2. Use Updated CSS Styles for the Toggle Switch

   Replace the CSS styles with the following concise and efficient code:

   /* Toggle Switch Styles */
   .switch {
     position: relative;
     display: inline-block;
     width: 30px;
     height: 14px;
     vertical-align: middle;
   }
   
   .switch input {
     opacity: 0;
     width: 0;
     height: 0;
   }
   
   .slider {
     position: absolute;
     cursor: pointer;
     background-color: #ccc;
     border-radius: 34px;
     top: 0;
     left: 0;
     right: 0;
     bottom: 0;
     transition: background-color 0.4s;
   }
   
   .slider:before {
     position: absolute;
     content: "";
     height: 10px;
     width: 10px;
     left: 2px;
     bottom: 2px;
     background-color: white;
     border-radius: 50%;
     transition: transform 0.4s;
   }
   
   /* When the checkbox is checked */
   input:checked + .slider {
     background-color: #2196F3;
   }
   
   input:checked + .slider:before {
     transform: translateX(16px);
   }
   
   /* Optional: Focus styles */
   input:focus + .slider {
     box-shadow: 0 0 2px #2196F3;
   }
   
   /* Rounded sliders */
   .slider {
     border-radius: 34px;
   }
   
   .slider:before {
     border-radius: 50%;
   }
   

   This set of styles ensures the toggle switch remains compact and visually appealing.

3. Mark Developer-Specific Content

   Wrap developer-specific content in a container with the class dev-only to control its visibility:

   <span class="dev-only">
     <a href="./link.html" style="color: white; text-decoration: none;">Link</a>
     <!-- Other dev-only elements -->
   </span>
   

   This step ensures content visibility responds to the toggle state.

4. Simplify JavaScript Logic

   Update the JavaScript to include a brief updateArrows function, focusing on the toggle functionality and essential arrow updates:

   <script>
     // Function to update Dev Mode and save the state to localStorage
     function updateDevMode() {
       const isDevMode = document.getElementById('devToggle').checked;
       const devElements = document.querySelectorAll('.dev-only');
       devElements.forEach(el => {
         el.style.display = isDevMode ? '' : 'none';
       });
       // Save the state to localStorage
       localStorage.setItem('devMode', isDevMode);
   
       // Show or hide arrows based on dev mode
       const arrowSVG = document.getElementById('arrow');
       if (arrowSVG) {
         arrowSVG.style.display = isDevMode ? '' : 'none';
       }
   
       // If in dev mode, calculate arrows; otherwise, skip
       if (isDevMode) {
         updateArrows();
       }
     }
   
     // Initialize the toggle state based on localStorage
     function initializeDevMode() {
       const savedDevMode = localStorage.getItem('devMode');
       const isDevMode = savedDevMode === 'true'; // Convert string to boolean
       document.getElementById('devToggle').checked = isDevMode;
       updateDevMode(); // Apply the state
     }
   
     // Add event listener for toggle switch
     document.getElementById('devToggle').addEventListener('change', updateDevMode);
   
     // Function to update arrows (simplified)
     function updateArrows() {
       // Core logic to adjust arrow visibility and positions
       const arrowLine = document.getElementById("arrow-line");
       const deviceInterface = document.getElementById("device-interface");
       const iosDevices = document.getElementById("ios-devices");
   
       if (!deviceInterface || !iosDevices) {
         if (arrowLine) arrowLine.style.display = 'none';
         return;
       }
   
       if (arrowLine) arrowLine.style.display = ''; // Ensure arrow visibility if elements exist
     }
   
     // Initialize on page load
     window.onload = initializeDevMode;
   </script>
   

   This streamlined approach focuses on visibility toggling and essential adjustments, avoiding unnecessary complexity.

Storing the Toggle State for Persistent Dev Mode

To ensure that the developer mode preference persists across browser sessions and new tabs, the toggle state is stored using the browser's localStorage. This approach allows the application to remember the user's choice even after closing and reopening the browser:

1. Saving the Toggle State:
   • When the toggle switch is changed, the updateDevMode function saves the current state (true for dev mode enabled, false for disabled) to localStorage using localStorage.setItem('devMode', isDevMode);.
2. Initializing the Toggle State on Page Load:
   • Upon loading the page, the initializeDevMode function retrieves the saved state from localStorage using localStorage.getItem('devMode');.
   • The retrieved value is a string, so it is compared to 'true' to convert it to a boolean.
   • The toggle switch is then set to the retrieved state, and updateDevMode is called to apply the visibility settings.
3. Default State When localStorage Is Cleared:
   • If localStorage is cleared (e.g., by deleting the cache), there will be no saved state.
   • In this case, the toggle switch defaults to not dev mode, as the checked attribute is removed from the HTML input.
   • The initializeDevMode function sets the toggle to false (not dev mode) when no saved state is found.

<script>
  // Function to update Dev Mode and save the state to localStorage
  function updateDevMode() {
    const isDevMode = document.getElementById('devToggle').checked;
    const devElements = document.querySelectorAll('.dev-only');
    devElements.forEach(el => {
      el.style.display = isDevMode ? '' : 'none';
    });
    // Save the state to localStorage
    localStorage.setItem('devMode', isDevMode);

    // Show or hide arrows based on dev mode
    const arrowSVG = document.getElementById('arrow');
    if (arrowSVG) {
      arrowSVG.style.display = isDevMode ? '' : 'none';
    }

    // If in dev mode, calculate arrows; otherwise, skip
    if (isDevMode) {
      updateArrows();
    }
  }

  // Initialize the toggle state based on localStorage
  function initializeDevMode() {
    const savedDevMode = localStorage.getItem('devMode');
    const isDevMode = savedDevMode === 'true'; // Convert string to boolean
    document.getElementById('devToggle').checked = isDevMode;
    updateDevMode(); // Apply the state
  }

  // Add event listener for toggle switch
  document.getElementById('devToggle').addEventListener('change', updateDevMode);

  // Function to update arrows (simplified)
  function updateArrows() {
    // Core logic to adjust arrow visibility and positions
    const arrowLine = document.getElementById("arrow-line");
    const deviceInterface = document.getElementById("device-interface");
    const iosDevices = document.getElementById("ios-devices");

    if (!deviceInterface || !iosDevices) {
      if (arrowLine) arrowLine.style.display = 'none';
      return;
    }

    if (arrowLine) arrowLine.style.display = ''; // Ensure arrow visibility if elements exist
  }

  // Initialize on page load
  window.onload = initializeDevMode;
</script>

Behavior on Cache Reset

Default State: By removing the checked attribute from the HTML checkbox, the default state when localStorage is cleared is not dev mode.

Persistence: When the browser cache is not cleared, the toggle state persists across sessions and new tabs, reflecting the user's last choice.

Initialization Logic Correction

1. Incorrect: const isDevMode = savedDevMode === 'false';
2. Corrected: const isDevMode = savedDevMode === 'true';

This correction ensures that when localStorage contains 'true', isDevMode is set to true, enabling dev mode. Conversely, if localStorage contains 'false' or is null, isDevMode is set to false, disabling dev mode.

Resolving Asynchronous Loading Issues in Web Applications Using jQuery and Local Storage

Understanding the Problem

When utilizing jQuery's .load() method to inject external HTML content, such as link.html, into a web page, inconsistencies may arise. This typically occurs because scripts that interact with elements within the loaded content execute before those elements are fully integrated into the Document Object Model (DOM). As a result, elements like devToggle might not be recognized, leading to unpredictable behavior.

Solution Overview

To address the aforementioned issue, the following steps are recommended:

1. Ensure jQuery is Loaded Before Use: The jQuery library must be included prior to any scripts that depend on it.
2. Utilize the .load() Callback Function: Incorporate initialization and event listener attachment within the callback function of the .load() method to ensure elements are available in the DOM.
3. Adjust JavaScript Functions: Modify initialization and event listener functions to operate after the content has been loaded.
4. Ensure All Dependent Code Runs After Loading: Any scripts that interact with dynamically loaded content should execute only after those elements are fully loaded.

Implementation Steps

1. Include jQuery Before Using It

It is imperative to include the jQuery library before any scripts that rely on it. This ensures that jQuery functions are available when needed.

<!-- Include jQuery first -->
<script src="jquery-3.6.0.min.js"></script>

2. Use the .load() Callback Function

Modify the .load() method to include a callback function. This callback executes after the external content has been successfully loaded, ensuring that the elements are present in the DOM before any interactions occur.

$(function() {
    $("#common-html").load("link.html", function() {
        // Initialization code goes here
        initializeDevMode();

        // Add event listener for the toggle switch
        const devToggle = document.getElementById('devToggle');
        if (devToggle) {
            devToggle.addEventListener('change', updateDevMode);
        } else {
            console.error('devToggle element not found.');
        }
    });
});

3. Adjust JavaScript Functions

Remove any event listeners or initialization calls that occur before the content is loaded. Ensure that functions like initializeDevMode() and updateDevMode() are defined and accessible.

// Define the updateDevMode function
function updateDevMode() {
    const isDevMode = document.getElementById('devToggle').checked;
    console.log('Dev Mode toggled:', isDevMode);
    const devElements = document.querySelectorAll('.dev-only');
    devElements.forEach(el => {
        el.style.display = isDevMode ? '' : 'none';
    });
    // Save the state to localStorage
    localStorage.setItem('devMode', isDevMode);

    // Show or hide arrows based on dev mode
    const arrowSVG = document.getElementById('arrow');
    if (arrowSVG) {
        arrowSVG.style.display = isDevMode ? '' : 'none';
    }

    // If in dev mode, calculate arrows; otherwise, skip
    if (isDevMode) {
        updateArrows();
    }
}

// Define the initializeDevMode function
function initializeDevMode() {
    const savedDevMode = localStorage.getItem('devMode');
    const isDevMode = savedDevMode === 'true';
    console.log('Initializing Dev Mode:', isDevMode);
    const devToggle = document.getElementById('devToggle');
    if (devToggle) {
        devToggle.checked = isDevMode;
        updateDevMode();
    } else {
        console.error('devToggle element not found during initialization.');
    }
}

// Define the updateArrows function
function updateArrows() {
    // Arrow updating logic...
}

Dynamically Updating Content Based on Dev Mode Toggle in JavaScript

In web development, adjusting the user interface based on specific modes or settings, such as a development mode (dev mode), is a common practice. This enhances user experience by providing additional information or functionality when necessary. The following outlines two methods to dynamically update webpage content to display "Echo" when not in dev mode and "Echocardiography" when in dev mode. Both methods utilize JavaScript to manipulate DOM elements based on the state of a dev mode toggle.

Method I: Updating Text Content Using JavaScript

HTML Update

Begin by ensuring the HTML structure includes an element with an identifiable id for the link text:

<td align="center" style="background-color: rgba(224, 223, 248, 0.3);">
  <span id="echo-devices">
    <b><a href="./echocardiography.html" id="echo-link">Echocardiography</a></b>
  </span>
</td>

In this snippet, the id="echo-link" attribute is added to the <a> tag, allowing for direct manipulation of its text content.

JavaScript Update

Within the updateDevMode function, add logic to update the text content of the link based on the dev mode state:

function updateDevMode() {
  const isDevMode = document.getElementById('devToggle').checked;
  console.log('Dev Mode toggled:', isDevMode);

  const devElements = document.querySelectorAll('.dev-only');
  devElements.forEach(el => {
    el.style.display = isDevMode ? '' : 'none';
  });

  // Update Echo text based on dev mode
  const echoLink = document.getElementById('echo-link');
  echoLink.textContent = isDevMode ? "Echocardiography" : "Echo";

  // Save the state to localStorage
  localStorage.setItem('devMode', isDevMode);

  // Show or hide arrows based on dev mode
  const arrowSVG = document.getElementById('arrow');
  if (arrowSVG) {
    arrowSVG.style.display = isDevMode ? '' : 'none';
  }

  // If in dev mode, calculate arrows; otherwise, skip
  if (isDevMode) {
    updateArrows();
  }
}

Explanation

• Dynamic Text Update: The textContent property of the echo-link element is updated depending on the isDevMode state. This changes the displayed link text to "Echocardiography" when in dev mode and "Echo" when not.
• Preservation of Link Functionality: By updating the textContent rather than replacing the entire element, the link's functionality and attributes remain intact.
• Minimal Code Addition: This method introduces minimal additional code, ensuring simplicity and maintainability.

Behavior

1. Dev Mode Enabled: The link displays as Echocardiography.
2. Dev Mode Disabled: The link displays as Echo.

This approach provides a seamless user experience by dynamically adjusting the link text with minimal impact on the existing codebase.

Method II: Modifying Inner HTML to Include Conditional Line Break

HTML Structure

The HTML remains the same as in Method I:

<td align="center" style="background-color: rgba(224, 223, 248, 0.3);">
  <span id="echo-devices">
    <b><a href="./echocardiography.html">Echocardiography</a></b>
  </span>
</td>

JavaScript Update

Modify the updateDevMode function to adjust the inner HTML of the echo-devices element:

function updateDevMode() {
  const isDevMode = document.getElementById('devToggle').checked;
  console.log('Dev Mode toggled:', isDevMode);

  const devElements = document.querySelectorAll('.dev-only');
  devElements.forEach(el => {
    el.style.display = isDevMode ? '' : 'none';
  });

  // Update Echo text and line break for dev mode
  const echoDevices = document.getElementById('echo-devices');
  if (isDevMode) {
    echoDevices.innerHTML = '<b><a href="./echocardiography.html">Echocardiography</a></b><br>';
  } else {
    echoDevices.innerHTML = '<b><a href="./echocardiography.html">Echo</a></b>';
  }

  // Save the state to localStorage
  localStorage.setItem('devMode', isDevMode);

  // Show or hide arrows based on dev mode
  const arrowSVG = document.getElementById('arrow');
  if (arrowSVG) {
    arrowSVG.style.display = isDevMode ? '' : 'none';
  }

  // If in dev mode, calculate arrows; otherwise, skip
  if (isDevMode) {
    updateArrows();
  }
}

Explanation

• Dynamic Inner HTML Update: The innerHTML property of the echo-devices element is updated based on the dev mode state.
• Conditional Line Break: In dev mode, a <br> tag is added after the link, potentially altering the layout as needed. When not in dev mode, the <br> tag is omitted.
• Text Change: The link text is adjusted between "Echocardiography" and "Echo", matching the dev mode state.
• Complete Content Refresh: By updating the innerHTML, the entire content of the echo-devices element is refreshed, ensuring that any additional HTML elements (like the <br> tag) are accurately rendered.

Behavior

1. Dev Mode Enabled:

   <b><a href="./echocardiography.html">Echocardiography</a></b><br>

2. Dev Mode Disabled:

   <b><a href="./echocardiography.html">Echo</a></b>

This method provides more control over the HTML content, allowing for structural changes such as adding or removing line breaks in addition to changing the text.

link.html

Text scaling issue and fix in v2.1.5 (Written November 27, 2025)

I. Overview of the observed issue

In earlier versions of the dashboard, the layout was tuned primarily on desktop browsers. When the same page was opened in iPhone Safari, some text blocks appeared larger, while others appeared smaller than expected. This inconsistent scaling caused the visual balance and alignment of the dashboard to be disturbed on mobile devices, even though the underlying HTML and CSS were identical.

The symptoms observed on iPhone Safari can be summarized as follows:

• Certain text (for example, within some table cells or heading-like areas) appeared noticeably enlarged compared to desktop rendering.
• Other text remained at the originally defined size or even appeared relatively smaller in comparison.
• Because different blocks were scaled differently, the carefully aligned columns and rows in the dashboard no longer matched the desktop appearance.

II. Cause at the browser level

This behavior is caused by mobile browsers, particularly iOS Safari, performing automatic “text inflation” or “text-size adjustment” to enhance readability on small screens. When a page uses relatively small font sizes or sits inside narrow containers, iOS Safari may selectively increase the font size of certain blocks, while leaving others unchanged. As a result, the ratios that were carefully designed on desktop are no longer preserved on mobile.

CSS provides a dedicated property, text-size-adjust (and its vendor-prefixed form -webkit-text-size-adjust), which controls this behavior. By default, the browser behaves as though this property is set to auto, allowing heuristic adjustments. Explicitly setting this value to a fixed percentage such as 100% instructs the browser to render text at the declared size, rather than applying additional inflation heuristics per block.

III. Changes introduced in version 2.1.5

In version 2.1.5, the codebase was updated in a minimal and targeted way to eliminate inconsistent text scaling on iPhone Safari, while preserving the existing desktop layout. The key changes are:

• Version metadata updated from 2.1.4 to 2.1.5 in the CSS header and footer badge.
• A global rule was added on the html element to fix the text-size adjustment to 100%, preventing iOS Safari from inflating some text blocks relative to others.
• A normalization rule was added for the body element to remove the default browser margin, ensuring that the main layout container sits consistently against the viewport edge across devices.

/* =========================================================
   Version 2.1.4 – Layout, visuals, and helpers
   ========================================================= */

/* =========================================================
   Version 2.1.5 – Layout, visuals, and helpers
   ========================================================= */

/* Prevent iOS Safari from auto-adjusting text sizes */
html {
  -webkit-text-size-adjust: 100%;
  text-size-adjust: 100%;
}

body {
  margin: 0;
}

IV. Detailed before-and-after snippets

The following snippets highlight precisely what changed at the top of the CSS block between version 2.1.4 and version 2.1.5. Only the parts relevant to text scaling and version labeling are shown.

Before (v2.1.4, top of the CSS section)

/* =========================================================
   Version 2.1.4 – Layout, visuals, and helpers
   ========================================================= */

/* ---------- Common effects ---------- */
@keyframes blink {
  0%,
  100% {
    opacity: 1;
  }
  50% {
    opacity: 0;
  }
}

.blink-text {
  animation: blink 2s infinite;
}

After (v2.1.5, top of the CSS section)

/* =========================================================
   Version 2.1.5 – Layout, visuals, and helpers
   ========================================================= */

/* Prevent iOS Safari from auto-adjusting text sizes */
html {
  -webkit-text-size-adjust: 100%;
  text-size-adjust: 100%;
}

body {
  margin: 0;
}

/* ---------- Common effects ---------- */
@keyframes blink {
  0%,
  100% {
    opacity: 1;
  }
  50% {
    opacity: 0;
  }
}

.blink-text {
  animation: blink 2s infinite;
}

All other CSS and HTML structures, including the three-way toggle, navigation table, device column, and SVG arrows, remain unchanged between v2.1.4 and v2.1.5. The update is intentionally minimal and focused on ensuring consistent typography across desktop and iPhone Safari.

V. Expected effect on layout and readability

With the global text-size-adjust configuration in place, iPhone Safari now renders font sizes according to the explicit values defined in the CSS, rather than applying additional, block-specific scaling. The main consequences for the dashboard are:

• Text that previously appeared enlarged relative to its neighbors on iPhone now matches the intended proportions seen on desktop.
• Sections that looked “shrunk” compared to adjacent blocks now align properly in terms of font size.
• The visual relationships between headings, body text, table contents, and labels are preserved across devices, improving the consistency of the user experience.

Line wrapping on small screens will naturally differ from desktop due to the narrower viewport, but the relative sizes of text elements now remain faithful to the original design intent. This addresses the disruption in layout that was observed when viewing the dashboard on iPhone Safari.

VI. Additional centering fix in version 2.1.6

After the introduction of the text scaling fix in version 2.1.5, an additional layout side effect was observed: the page appeared visually left-aligned, especially noticeable when viewed on both desktop and iPhone. This perception was primarily due to the removal of the browser’s default body margin, which caused the main content to sit flush with the left edge of the viewport.

What was changed (before / after)

1. Restored centered feel by removing the body margin override

   In version 2.1.5, a global reset on the <body> element was introduced to normalize spacing across devices:

   Before (v2.1.5):

   /* ---------- Mobile text scaling control (iOS Safari, etc.) ---------- */
   html {
     -webkit-text-size-adjust: 100%;
     text-size-adjust: 100%;
   }
   
   body {
     margin: 0;
   }

   The body { margin: 0; } rule removed the browser’s default outer margin, which visually pushed the entire layout flush to the left edge of the viewport on both desktop and iPhone, creating the impression that the page was no longer “centered”.

   After (v2.1.6):

   /* ---------- Mobile text scaling control (iOS Safari, etc.) ---------- */
   html {
     -webkit-text-size-adjust: 100%;
     text-size-adjust: 100%;
   }

   In version 2.1.6, the body { margin: 0; } block is removed. The browser’s default margin is restored, so the main content no longer hugs the left edge and instead sits with the same visual centering and outer margins as in earlier versions, while still keeping the text scaling fix on the html element.

   In addition, the main container is explicitly constrained and centered, for example by using a horizontally centered margin and a maximum width (such as max-width on the primary .main wrapper). This reinforces the centered “card” feel on larger screens without disturbing the behavior on narrow mobile viewports.

2. Version identifier update

   To reflect the new adjustment, the version metadata is incremented from 2.1.5 to 2.1.6 in both the CSS header comment and the footer badge.

   Before (v2.1.5, CSS header):

   /* =========================================================
      Version 2.1.5 – Layout, visuals, and helpers
      ========================================================= */

   After (v2.1.6, CSS header):

   /* =========================================================
      Version 2.1.6 – Layout, visuals, and helpers
      ========================================================= */

   Before (v2.1.5, footer badge):

   <div class="version-info">
     <div>v2.1.5 • 27 Nov 2025</div>
     <div style="font-size: 9px;">
       © 2013 nGene Hemodynamic Research Center. All rights reserved.
     </div>
   </div>

   After (v2.1.6, footer badge):

   <div class="version-info">
     <div>v2.1.6 • 27 Nov 2025</div>
     <div style="font-size: 9px;">
       © 2013 nGene Hemodynamic Research Center. All rights reserved.
     </div>
   </div>

   These changes ensure that the version label displayed in the interface accurately reflects the underlying layout and behavior implemented in the code.

Only this HTML file is changed between versions 2.1.5 and 2.1.6. The mobile text scaling fix remains active via the html { -webkit-text-size-adjust: 100%; text-size-adjust: 100%; } rule, while the visual centering is restored and strengthened by allowing the browser’s default body margins to apply again and by constraining and centering the main container. No other structural or behavioral changes are introduced.

Written on November 27, 2025

Python Script

Converting HTML Tables to Excel with Python

Automating the conversion of HTML tables to Excel spreadsheets can streamline data analysis and reporting tasks. This guide demonstrates how to use Python's pandas library to parse HTML content, transform it into a DataFrame, and export it as an Excel file on the user's Desktop, ensuring compatibility across different operating systems.

• Cross-Platform Compatibility: Automatically detects the user's Desktop directory on both macOS and Windows.
• Error Handling: Manages potential issues during the conversion and export process.
• Code Clarity and Maintainability: Well-documented code with descriptive comments for ease of understanding.

Python Script:

Below is the refined Python script tailored to perform the HTML to Excel conversion. The HTML content is represented by a placeholder (~~~) for brevity.

import pandas as pd
from pathlib import Path
import os

# The HTML content as a string
html_content = """
~~~
"""

def get_desktop_path():
    """
    Returns the path to the user's Desktop in a cross-platform manner.
    """
    home = Path.home()
    desktop = home / 'Desktop'
    
    if desktop.exists() and desktop.is_dir():
        return desktop
    else:
        # Handle cases where Desktop might be localized or missing
        # Attempt to retrieve from environment variables
        if os.name == 'nt':  # For Windows
            desktop = Path(os.path.join(os.environ.get('USERPROFILE'), 'Desktop'))
        else:  # For macOS and Linux
            desktop = Path(os.path.join(os.environ.get('HOME'), 'Desktop'))
        
        if desktop.exists() and desktop.is_dir():
            return desktop
        else:
            # As a fallback, use the home directory
            return home

def main():
    try:
        # Read the HTML into pandas
        df_list = pd.read_html(html_content)  # This returns a list of DataFrames
        if not df_list:
            print("No tables found in the provided HTML content.")
            return
        df = df_list[0]  # Assuming there's only one table, it's the first item
        
        # Get the Desktop path
        desktop_path = get_desktop_path()
        
        # Define the output file path
        output_file = desktop_path / "converted_table.xlsx"
        
        # Export to Excel
        df.to_excel(output_file, index=False)
        
        print(f"Excel file successfully saved to: {output_file}")
    
    except Exception as e:
        print(f"An error occurred: {e}")

if __name__ == "__main__":
    main()

Written on November 12th, 2024

modify_tmp_html.py usage guide (Written May 13, 2025)

Purpose

The script modify_tmp_html.py streamlines post‑processing of a draft tmp.html file by inserting a decorative divider, adding an anchor for internal linking, stamping the document with “Written …” metadata, and appending a closing signature. These adjustments ensure consistent branding and navigation across locally generated HTML notes.

Prerequisites

• Python ≥ 3.8 with the datetime, pathlib, and re standard libraries (no external packages required).
• A directory that contains both modify_tmp_html.py and tmp.html.
• Write permission for overwriting tmp.html.
• Terminal access for executing the script with optional arguments.

Execution procedure

1. Navigate to the directory that hosts modify_tmp_html.py and tmp.html.
2. Invoke the script:
   python3 modify_tmp_html.py <keyword>
   The keyword is optional; when omitted, the first word of the first <h2> heading becomes the anchor identifier.
3. Verify the console message [OK] Updated: tmp.html to confirm successful completion.

Command‑line keyword parameter

An explicit keyword refines the anchor ID that precedes the main content. Supplying a descriptive noun improves URL‑style hashes when multiple entries are aggregated in a single log.

Internal processing flow

Output structure

After execution the document begins with a minimalistic divider block, followed by a left‑aligned backlink that mirrors the original title, and an invisible anchor padded by seven spaces. The first <h2> heading now contains the “Written …” suffix, and the trailing footer displays the same date in cursive form to close the article gracefully.

Typical example

$ cd /path/to/note
$ python3 modify_tmp_html.py Overview
[OK] Updated: /path/to/note/tmp.html

Troubleshooting notes

• [ERROR] File not found: indicates an incorrect working directory or missing tmp.html.
• [ERROR] No <h2> … </h2> block found suggests a malformed file header; ensure that the first heading is an <h2> element.
• Anchor duplication is avoided by embedding the current date prefix (YYYYMMDD_) in the ID.

Source code listing

# !/usr/bin/env python3
"""
modify_tmp_html.py  (refined -- May 1 2025)
-------------------------------------------
• Prepends a decorative divider block
• Inserts an anchor + 7 spaces
• Adds “Written …” markers and backlink
• Drops two blank lines after the first <h2>…</h2>
• Uses tmp.html that lives **next to this script**, not on the Desktop
• Lets the user supply an explicit keyword via the first CLI argument
"""

import datetime as _dt
import os as _os
import re as _re
import sys as _sys
from pathlib import Path

# ---------------------------------------------------------------------------

DIVIDER_BLOCK = (
    "</div>\n"
    "<!----------------------------------------------------------------------------\n"
    '  <hr style="border-top: 1px dashed #8c8b8b; margin: 20px 0;" />\n'
    "<!----------------------------------------------------------------------------><br/>\n"
    "<!---------------------------------------------------------------------------->\n"
    '<div class="main">\n\n'  # two line-breaks after the opening <div>
)

def main() -> None:
    # 1. Locate tmp.html **in the same directory as this script**
    script_dir = Path(__file__).resolve().parent
    html_path  = script_dir / "tmp.html"
    if not html_path.exists():
        _sys.exit(f"[ERROR] File not found: {html_path}")

    original = html_path.read_text(encoding="utf-8")

    # 2. Date fragments
    today          = _dt.date.today()                       # 2025-05-01
    ymd_           = today.strftime("%Y%m%d_")              # 20250501_
    month_day      = today.strftime("%B {d}, %Y").format(d=today.day)  # May 1, 2025
    written_suffix = f" (Written {month_day})"

    # 3. Capture first <h2>…</h2>
    h2_match = _re.search(r"<h2>(.*?)</h2>", original, flags=_re.DOTALL | _re.IGNORECASE)
    if not h2_match:
        _sys.exit("[ERROR] No <h2> … </h2> block found in the file.")

    title_plain = _re.sub(r"<[^>]+>", "", h2_match.group(1)).strip()

    # --- NEW: explicit keyword from CLI (fallback to first word) ------------
    if len(_sys.argv) > 1 and _sys.argv[1].strip():
        keyword = _sys.argv[1].strip()
    else:
        keyword = _re.search(r"\b\w+\b", title_plain).group(0)

    anchor_id = f"{ymd_}{keyword}"

    # 4. Add “Written …” inside the first <h2> and drop two blank lines after it
    def add_written(match: _re.Match) -> str:
        inner = match.group(1).rstrip()
        return f"<h2>{inner}{written_suffix}</h2>\n"

    modified = _re.sub(
        r"<h2>(.*?)</h2>",
        add_written,
        original,
        count=1,
        flags=_re.DOTALL | _re.IGNORECASE,
    )

    # 5. Assemble the new head section
    anchor_tag = f'<a id="{anchor_id}"></a>' + " " * 7  # 7 blank spaces after anchor

    head_block = (
        DIVIDER_BLOCK
        + f'<p style="margin-left: 10px; text-align: left;"><a href="#{anchor_id}">{title_plain}{written_suffix}</a></p>\n'
        + anchor_tag
    )

    # 6. Append closing “Written on …” paragraph and backlink
    footer = f'\n\n<p style="text-align: right;"><i>Written on {month_day}</i></p>\n'

    # 7. Write back
    html_path.write_text(head_block + modified + footer, encoding="utf-8")
    print(f"[OK] Updated: {html_path}")

# ---------------------------------------------------------------------------

if __name__ == "__main__":
    main()

Written on May 13, 2025

pretiffy.tmp_html.py usage guide (Written May 13, 2025)

Purpose

The utility pretiffy.tmp_html.py post‑processes a working tmp.html file by converting *asterisk*‑wrapped phrases into HTML‑compliant bold tags, beautifying markup, collapsing multi‑line inline elements, trimming superfluous whitespace, and purging residual citation markers. The result is a cleaner, single‑line‑per‑paragraph document ready for archival or distribution.

Environment setup

# macOS example
brew install python
pip3 install --user beautifulsoup4

# optional virtual‑environment workflow
python3 -m venv venv
source venv/bin/activate
pip install beautifulsoup4

Execution procedure

1. Ensure pretiffy.tmp_html.py resides in the same directory as tmp.html.
2. Invoke the script:
   python3 pretiffy.tmp_html.py
3. Confirm the terminal output
   tmp.html updated (boldified, prettified, compact tags, trimmed spaces, citations removed).

Core transformations

• Asterisk boldification – converts *phrase* or **phrase** into <b>phrase</b>.
• HTML prettification – employs BeautifulSoup to indent elements predictably.
• Inline tag collapsing – rewrites <tag>\n  text\n</tag> as <tag>text</tag> when the content fits on one line.
• Whitespace trimming – removes excess spaces inside and after bold tags, preserving adjacent punctuation.
• Citation cleanup – strips patterns such as :contentReference[oaicite:47]{index=47}.

Internal processing flow

Output characteristics

The finished HTML contains single‑line paragraphs, compact inline tags, and precise bold markup without lingering citation placeholders. Indentation follows BeautifulSoup’s default two‑space style, facilitating diff reviews and version control.

Typical example

$ cd /projects/note
$ python3 pretiffy.tmp_html.py
tmp.html updated (boldified, prettified, compact tags, trimmed spaces, citations removed).

Troubleshooting notes

• ModuleNotFoundError: bs4 – BeautifulSoup is missing; install with pip install beautifulsoup4.
• UnicodeDecodeError – ensure tmp.html is UTF‑8 encoded.
• To undo formatting mishaps, restore the previous file version from version control or a manual backup.

Source code listing

# !/usr/bin/env python3
"""
pretiffy.tmp_html.py

brew install python
pip3 install --user beautifulsoup4
python3 -m venv venv
source venv/bin/activate
pip install beautifulsoup4
python3 pretiffy.tmp_html.py

--------------------------------------------------------------------
• Make *phrase* / **phrase** bold → <b>phrase</b>
• Prettify HTML
• Collapse any  <tag>\n  ...text...\n</tag>  →  <tag>...text...</tag>
  (paragraphs stay single-line too)
• Trim extra spaces:
    – <b> ~~~~ </b>     → <b>~~~~</b>
    – </b> ANY_SYMBOL   → </b>ANY_SYMBOL   (comma, period, ?, !, etc.)
• REMOVE citation markers such as
      :contentReference[oaicite:47]{index=47}
--------------------------------------------------------------------
"""

import re
from pathlib import Path
from bs4 import BeautifulSoup, NavigableString

HTML_FILE = Path("tmp.html")

# ----------------- helpers ----------------------------------------------------

ASTERISK_BOLD = re.compile(
    r"(?<!\*)"        # not preceded by another *
    r"(\*{1,2})"      # opening * or **
    r"(.+?)"          # minimal phrase
    r"\1"             # matching closing * or **
    r"(?!\*)"         # not followed by another *
    , flags=re.DOTALL
)

def to_bold(match: re.Match) -> str:
    """Return <b>phrase</b> for the matched *phrase* / **phrase**."""
    return f"<b>{match[2]}</b>"

def flatten_paragraphs(html: str) -> str:
    """Remove internal newlines/indentation inside <p>…</p> so each paragraph is one long line."""
    def repl(m: re.Match) -> str:
        open_tag, body, close_tag = m.groups()
        body_one_line = re.sub(r"\s*\n\s*", " ", body).strip()
        return f"{open_tag}{body_one_line}{close_tag}"

    return re.sub(r"(<p\b[^>]*>)([\s\S]*?)(</p>)",
                  repl, html, flags=re.IGNORECASE)

def inline_text_tags(html: str) -> str:
    """Collapse any tag whose body is a single line into that same single line."""
    pattern = re.compile(
        r"<(\w+)([^>]*)>\s*\n\s*([^\n]+?)\s*\n\s*</\1>",
        flags=re.MULTILINE
    )
    while True:               # repeat until no more collapsible tags
        new_html = pattern.sub(
            lambda m: f"<{m.group(1)}{m.group(2)}>{m.group(3).strip()}</{m.group(1)}>",
            html
        )
        if new_html == html:
            break
        html = new_html
    return html

def fix_bold_spacing(html: str) -> str:
    """
    • Trim spaces *inside* <b> … </b>
    • Remove spaces between </b> and any following punctuation/symbol.
      (e.g. '</b> ,' → '</b>,', '</b> ?' → '</b>?')
    """
    # 1) inside the tag
    html = re.sub(
        r"<b>\s*(.*?)\s*</b>",
        lambda m: f"<b>{m.group(1).strip()}</b>",
        html,
        flags=re.DOTALL
    )
    # 2) after the tag – match any char that is not alphanumeric or whitespace
    html = re.sub(r"</b>\s+([^\w\s])", r"</b>\1", html)
    return html

def remove_citations(html: str) -> str:
    """Remove citation markers such as :contentReference[oaicite:47]{index:47}."""
    pattern = re.compile(r":?contentReference\[oaicite:[^\]]+\]\{index=[^\}]+\}")
    return pattern.sub("", html)

# ----------------- main workflow ---------------------------------------------

def main() -> None:
    soup = BeautifulSoup(HTML_FILE.read_text(encoding="utf-8"), "html.parser")

    # 1) Boldify every *…* / **…** occurrence inside plain-text nodes
    for node in soup.find_all(string=True):
        if isinstance(node, NavigableString):
            new_html = ASTERISK_BOLD.sub(to_bold, node)
            if new_html != node:
                node.replace_with(BeautifulSoup(new_html, "html.parser"))

    # 2) Prettify, tidy paragraphs, inline simple tags, fix bold spacing, remove citations
    pretty_html = soup.prettify(formatter="html")
    pretty_html = flatten_paragraphs(pretty_html)
    pretty_html = inline_text_tags(pretty_html)
    pretty_html = fix_bold_spacing(pretty_html)
    pretty_html = remove_citations(pretty_html)

    HTML_FILE.write_text(pretty_html, encoding="utf-8")
    print("tmp.html updated (boldified, prettified, compact tags, trimmed spaces, citations removed).")

if __name__ == "__main__":
    main()

Written on May 13, 2025

Color dataset luminance‑sorter usage guide (Written May 13, 2025)

Purpose

The script reorders a JavaScript columns array so that each nested group of swatches progresses from dark to bright, measured by relative luminance. The sorted data set is emitted as a JavaScript constant ready for direct inclusion in front‑end projects.

Environment setup

• Python ≥ 3.8 (no external libraries required).
• Input file containing a literal assignment such as const columns = [ … ];.
• Optional command‑line arguments to select custom input and output paths.

Execution procedure

1. Place colorDataset.txt (or any suitably named file) in the working directory.
2. Run the script:
   python3 reorder_by_luminance.py [input] [output]
   When arguments are omitted, defaults are colorDataset.txt and colorDataset_sorted.txt.
3. Observe the success message
   ✔ Luminance‑sorted dataset written to colorDataset_sorted.txt

Luminance metric

Relative luminance is computed by 0.2126 R + 0.7152 G + 0.0722 B, following the sRGB standard that weights green most heavily, then red, then blue. Sorting by this value arranges colors in an optically coherent sequence from darkest to brightest within each column group.

Internal processing flow

Input and output conventions

The input file must declare columns = [ … ];. Any other content is ignored. The output reproduces the same constant assignment, with indentation preserved and elements neatly aligned for readability.

Typical example

$ python3 reorder_by_luminance.py
✔ Luminance‑sorted dataset written to colorDataset_sorted.txt

Troubleshooting notes

• IndexError: ‘NoneType’ object has no attribute ‘group’ – the input file lacks a columns = assignment.
• json.decoder.JSONDecodeError – trailing commas or unquoted keys remain; verify that the helper regexes match the file’s structure.
• Colors that appear out of order may use non‑numeric RGB values or strings instead of three integers.

Source code listing

# !/usr/bin/env python3
"""
Re‑order a JavaScript `columns` array by luminance (dark → bright).
"""

import json, re, sys
from pathlib import Path

LUM = lambda r, g, b: 0.2126*r + 0.7152*g + 0.0722*b    # relative luminance

# ── comment / comma cleaners ────────────────────────────────────────────────
def _strip_js_comments(text: str) -> str:
    text = re.sub(r"/\*[\s\S]*?\*/", "", text)          # /* … */
    return re.sub(r"//[^\n]*", "", text)                # // …

def _remove_trailing_commas(text: str) -> str:
    return re.sub(r",\s*([\]}])", r"\1", text)

# ── JS → Python and back ────────────────────────────────────────────────────
def js_columns_to_python(js_text: str):
    array_literal = re.search(r"columns\s*=\s*(\[[\s\S]*?]);", js_text).group(1)
    clean = _strip_js_comments(array_literal)
    clean = _remove_trailing_commas(clean)
    clean = re.sub(r"(\{|,)\s*(name|rgb)\s*:", r'\1 "\2":', clean)
    return json.loads(clean)

def python_to_js_columns(py_cols):
    js = "const columns = [\n"
    for group in py_cols:
        js += "  [\n"
        for swatch in group:
            n, (r, g, b) = swatch["name"], swatch["rgb"]
            js += f'    {{ name: "{n}", rgb: [{r}, {g}, {b}] }},\n'
        js = js.rstrip(",\n") + "\n  ],\n"
    return js.rstrip(",\n") + "\n];\n"

def reorder_columns(cols):
    return [sorted(g, key=lambda o: LUM(*o["rgb"])) for g in cols]

# ── main ────────────────────────────────────────────────────────────────────
def main():
    inp = Path(sys.argv[1] if len(sys.argv) > 1 else "colorDataset.txt")
    out = Path(sys.argv[2] if len(sys.argv) > 2 else "colorDataset_sorted.txt")

    cols_sorted = reorder_columns(js_columns_to_python(inp.read_text()))
    out.write_text(python_to_js_columns(cols_sorted), encoding="utf‑8")

    print(f"✔ Luminance‑sorted dataset written to {out}")

if __name__ == "__main__":
    main()

Written on May 13, 2025

Clean_urls_in_place.py usage guide (Written May 19, 2025)

The utility clean_urls_in_place.py removes all URL strings from an existing tmp.html file, collapses any double-spaces left behind, and writes the sanitised markup back to disk. The script assists in preparing documents for contexts where external links must be suppressed before publication or archival.

Environment setup

# macOS example
brew install python             # modern Python interpreter

# optional virtual-environment workflow
python3 -m venv venv
source venv/bin/activate
# no third-party packages required

Execution procedure

1. Place clean_urls_in_place.py in the same directory as tmp.html.
2. Run the script:
   $ python3 clean_urls_in_place.py
3. Observe the terminal confirmation:
   [*] tmp.html cleaned in place

Core transformations

• URL stripping – removes full http:// or https:// links, bare domains, and linked paths.
• Whitespace compaction – collapses two or more consecutive spaces into a single space to preserve neat formatting.

Internal processing flow

Output characteristics

The resulting HTML retains its original structure while omitting every web-site address. All formerly linked words remain intact, now free of hyperlinks. Excess internal spacing is normalised, producing a tidy file suitable for compliance reviews and version-control diffs.

Typical example

$ cd /projects/note
$ python3 clean_urls_in_place.py
[*] tmp.html cleaned in place

Troubleshooting notes

• FileNotFoundError – confirm that tmp.html exists in the script’s directory.
• PermissionError – check write permissions on tmp.html or run with elevated privileges if necessary.
• Encoding issues – ensure the source file is valid UTF-8 or convert it before running the script.
• Accidental overwrites can be reversed by restoring the previous revision from version control or a backup copy.

Source code listing

# !/usr/bin/env python3
"""
clean_urls_in_place.py – strip all web-site addresses out of tmp.html

Usage
-----
$ python clean_urls_in_place.py
(Re)writes tmp.html in place.
"""

import re
from pathlib import Path

# --- file location ----------------------------------------------------------
HERE = Path(__file__).resolve().parent
HTML_FILE = HERE / "tmp.html"

# --- regex for URLs / bare domains -----------------------------------------
url_pattern = re.compile(
    r"""
    (?:https?://)?            # optional http:// or https://
    (?:www\.)?                # optional www.
    (?:[a-zA-Z0-9-]+\.)+      # one or more sub-domains + dot
    [a-zA-Z]{2,}              # top-level domain (com, kr, tr, …)
    (?:/[^\s<>"']*)?          # optional path until whitespace or quote
    """,
    re.VERBOSE,
)

# --- process & overwrite ----------------------------------------------------
html_text = HTML_FILE.read_text(encoding="utf-8")
cleaned   = url_pattern.sub("", html_text)
cleaned   = re.sub(r"[ \t]{2,}", " ", cleaned)  # collapse leftover double-spaces

HTML_FILE.write_text(cleaned, encoding="utf-8")
print("[*] tmp.html cleaned in place")

Written on May 19, 2025

Converting MathJax-Rich HTML to Word (DOCX) with Python (Written July 15, 2025)

Need a single-command way to turn an HTML article full of MathJax equations into a Word document whose formulas open in Insert ▶ Equation mode? That’s exactly what 2docx.py (v 0.1.6-dev) does. Drop it next to any .html file and run—it spits out a fully-styled DOCX on A4 paper with 1 cm margins and no unwanted bold or colour quirks.

• Native Word equations: Converts \(...\), \[...\], $...$ and $$...$$ into editable OMML equations.
• Cross-platform desktop path: Works on macOS, Windows and Linux (no hard-coded paths).
• A4, narrow margins: Forces 1 cm on all sides via DOCX metadata—ready for journal submission.
• Style cleanup: • all-black text • strips any <b>/<strong> tags.
• Pandoc fallback: Uses system Pandoc if present, otherwise auto-downloads a private copy via pypandoc.
• Error messaging: Clear exits if Pandoc is missing or reference templates can’t be found.

Python Script (v 0.1.6-dev)

#!/usr/bin/env python3
"""
2docx.py — version 0.1.6-dev

Hyunsuk Frank Roh, MD
nGene Hemodynamic Research Center
July 15, 2025
© Hyunsuk Frank Roh 2025. All rights reserved.

Convert HTML containing MathJax/TeX into a DOCX file whose equations are
true Word “Insert ▶ Equation” objects.  Designed for A4 paper, 1 cm
margins, all-black text, and no bold styling.

# Installation (macOS example)
# ----------------------------
# 1. python3 -m venv venv && source venv/bin/activate
# 2. pip install --upgrade pip
# 3. pip install beautifulsoup4 pypandoc
# 4. brew install pandoc        # or: python3 -c "import pypandoc; pypandoc.download_pandoc()"
# 5. python3 2docx.py tmp.html  # → creates tmp.docx with editable equations
    
Usage
-----
    python 2docx.py [input.html]
    python 2docx.py report.html --reference template.docx
        # (optional) override styles with your own reference DOCX

Requirements
------------
* Python ≥ 3.8
* pip install beautifulsoup4
* Pandoc ≥ 2.17 on PATH  – or –  pip install pypandoc
"""

from __future__ import annotations
import re
import shutil
import subprocess
import sys
from pathlib import Path

from bs4 import BeautifulSoup

VERSION = "0.1.6-dev"
PANDOC_FMT = "html+tex_math_dollars+tex_math_single_backslash"

# --------------- TeX cleanup helpers ---------------
_space_before_sub = re.compile(r"\s+([_^])")
_text_macro = re.compile(r"\\text\{([^}]*)\}")

def tidy_tex(expr: str) -> str:
    """Remove stray spaces before _/^ and replace \\text with \\mathrm."""
    expr = _space_before_sub.sub(r"\1", expr)
    expr = _text_macro.sub(r"\\mathrm{\\1}", expr)
    return expr

# --------------- Pandoc helpers --------------------
def ensure_pandoc() -> bool:
    """Return True if system pandoc exists; else try pypandoc."""
    if shutil.which("pandoc"):
        return True
    try:
        import pypandoc  # type: ignore
        pypandoc.get_pandoc_version()
        return False
    except (ImportError, OSError):
        sys.exit("Pandoc not found. Install it or `pip install pypandoc`.")

def run_pandoc(cmd: list[str], system: bool) -> None:
    """Execute Pandoc via subprocess or pypandoc fallback."""
    if system:
        subprocess.run(cmd, check=True)
        return
    import pypandoc  # type: ignore
    out_idx = cmd.index("-o")
    pypandoc.convert_file(
        cmd[1],
        to="docx",
        format=PANDOC_FMT,
        outputfile=cmd[out_idx + 1],
        extra_args=cmd[2:out_idx] + cmd[out_idx + 2 :],
    )

# --------------- HTML preprocessing ----------------
def preprocess_html(html_path: Path) -> Path:
    """
    * Remove MathJax wrappers, leaving \\( … \\) or \\[ … \\]
    * Force heading colours to black
    """
    soup = BeautifulSoup(html_path.read_text(encoding="utf-8"), "html.parser")

    # Strip bold tags
    for bold in soup.find_all(["b", "strong"]):
        bold.unwrap()

    # Make all headings black
    for hdr in soup.find_all(re.compile(r"^h[1-6]$", re.I)):
        style = hdr.get("style", "")
        if "color" not in style.lower():
            style = (style + "; color:#000000").lstrip("; ")
        hdr["style"] = style

    for tag in soup.find_all("script"):
        tp = (tag.get("type") or "").lower()
        if not tp.startswith("math/tex"):
            continue
        tex = tidy_tex((tag.string or "").strip())
        block = f"\\[ {tex} \\]" if ("display" in tp or tag.get("mode") == "display") else f"\\( {tex} \\)"
        tag.replace_with(soup.new_string(block))

    # Drop external MathJax loaders
    for tag in soup.find_all("script", src=True):
        if "mathjax" in tag["src"].lower():
            tag.decompose()

    clean_path = html_path.with_suffix(".clean.html")
    clean_path.write_text(str(soup), encoding="utf-8")
    return clean_path

# --------------- DOCX conversion -------------------
def convert_to_docx(src: Path, dst: Path, use_system: bool, ref: Path | None) -> None:
    cmd = [
        "pandoc",
        str(src),
        "-f",
        PANDOC_FMT,
        "-t",
        "docx",
        "--standalone",
        "-o",
        str(dst),
        "--metadata=docx-page-size:A4",
        "--metadata=docx-page-margin-top:1cm",
        "--metadata=docx-page-margin-bottom:1cm",
        "--metadata=docx-page-margin-left:1cm",
        "--metadata=docx-page-margin-right:1cm",
    ]
    if ref:
        cmd += ["--reference-doc", str(ref.resolve())]
    run_pandoc(cmd, use_system)

# --------------- CLI entry -------------------------
def main() -> None:
    use_system = ensure_pandoc()
    args = sys.argv[1:]

    ref: Path | None = None
    if "--reference" in args:
        i = args.index("--reference")
        if i + 1 >= len(args):
            sys.exit("Error: --reference requires a .docx path.")
        ref = Path(args[i + 1]).expanduser().resolve()
        if not ref.exists():
            sys.exit(f"Reference DOCX '{ref}' not found.")
        del args[i : i + 2]

    if len(args) > 1:
        sys.exit("Usage:  python 2docx.py [input.html] [--reference template.docx]")

    html_file = Path(args[0]) if args else Path(__file__).with_name("tmp.html")
    if not html_file.exists():
        sys.exit(f"Input '{html_file}' not found.")

    cleaned = preprocess_html(html_file)
    try:
        output_docx = html_file.with_suffix(".docx")
        convert_to_docx(cleaned, output_docx, use_system, ref)
        print(f"✓ DOCX created: {output_docx}")
        print("  Open in Microsoft Word — equations are fully editable.")
    finally:
        cleaned.unlink(missing_ok=True)

if __name__ == "__main__":
    main()

Written on July 15, 2025

Why python -m is necessary in certain cases (Written October 31, 2025)

I. The core problem

Executing a file inside a package with python path/to/file.py often breaks intra-package imports or produces hard-to-trace behavior. The root is that direct execution does not run the code as part of its package; it runs it as a top-level script with __name__ == "__main__", altering import resolution and package context.

Problem statement. A module inside a package needs to import siblings (e.g., from . import utils or import utils), but direct execution causes ModuleNotFoundError, duplicated modules in memory, or surprising side effects because the interpreter does not treat the module as belonging to its package.

# Layout
project/
└── app/
    ├── __init__.py
    ├── main.py        # imports utils
    └── utils.py

# Direct execution (problem)
$ python app/main.py
Traceback (most recent call last):
  ModuleNotFoundError: No module named 'utils'

# As a module (solution)
$ python -m app.main
# works: 'app' is a package; 'utils' is resolved inside it

II. Why it breaks (root causes)

III. Common failure patterns

• ModuleNotFoundError for siblings: from . import utils or import utils fails because the file is not in a package context.
• Duplicated modules in memory: importing the package elsewhere yields two copies (__main__ vs pkg.mod), causing diverging global state.
• Relative import confusion: from ..core import api raises ImportError: attempted relative import with no known parent package.
• Shadowing standard library or third-party modules: running from a directory named like a library (e.g., logging/, requests/) hides the real package.
• Test/CLI tools behave differently: pytest, unittest, and internal CLIs rely on package semantics that direct execution bypasses.
• Multiprocessing on platforms using spawn: worker import paths fail unless the entry point is importable as a module.

IV. Problem-solving playbook

1. Prefer module mode for anything inside a package

   # Good
   python -m app.main
   python -m app.tools.migrate
   python -m app.scripts.seed_db
   

2. Add a package entry point with __main__.py (optional, when running the package itself)

   # app/__main__.py
   from .main import run
   
   if __name__ == "__main__":
       run()
   

   # Run the whole package as an app
   python -m app
   

3. Use absolute imports inside packages

   # Inside app/main.py
   # Prefer:
   from app import utils
   
   # If relative is used, it only works in module mode:
   from . import utils
   

4. Keep a clean project root on sys.path

   • Run commands from the project root.
   • Install the project in editable mode for tools and notebooks: pip install -e .
   • Avoid running from directories whose names shadow real modules.

5. Standardize CLIs and tests to module mode

   • CLIs: python -m app.cli, not python app/cli.py.
   • Tests: python -m pytest or python -m unittest to ensure consistent import context.

6. Make the entry point explicit

   # app/main.py
   def main():
       ...
   
   if __name__ == "__main__":
       # Allows direct 'python app/main.py' during quick prototyping,
       # but production usage should still prefer 'python -m app.main'.
       main()
   

V. Quick decision checklist

• Is the code inside a package directory (has __init__.py)? Use python -m.
• Does the code import siblings or parents? Use python -m and prefer absolute imports.
• Will the code be part of a CLI, test runner, or multiprocessing worker? Use python -m.
• Is reproducibility across environments important? Use python -m to stabilize import paths.

VI. Advanced scenarios and guidance

• Namespace packages (pkgutil/PEP 420): module mode avoids surprises when packages span multiple directories.
• Zipapps and frozen apps: consistent import semantics are crucial; structure for module execution.
• Multiprocessing (spawn): workers re-import the entry module; module mode ensures importability.
• Vendor or editable installs: rely on importable package names; avoid ad-hoc script paths.
• Standard library patterns: many tools ship entry points designed for -m (e.g., http.server, venv, unittest, pip).

VII. Minimal templates

Layout A: package with module entry

project/
└── app/
    ├── __init__.py
    ├── __main__.py   # optional for 'python -m app'
    ├── main.py       # 'python -m app.main'
    └── utils.py

Layout B: scripts kept under package

project/
└── app/
    ├── __init__.py
    ├── scripts/
    │   └── seed_db.py   # run as 'python -m app.scripts.seed_db'
    └── core/
        └── utils.py

VIII. Concise summary

Direct execution treats the file as a standalone script and discards the package context, which changes __name__, import paths, and module identity. This leads to failed relative imports, duplicated modules, and brittle behavior. Running with python -m executes code as part of its package, stabilizing imports, avoiding duplication, and making CLIs, tests, and multiprocessing robust across environments. Adopting module mode, absolute imports, and clear entry points resolves the class of problems systematically.

Written on October 31, 2025

Python Script

Generating Bitcoin Keypairs and 24-Word BIP39 Mnemonics with Python

Generating a Bitcoin keypair (secp256k1 private/public keys) and exporting derived addresses in common formats can support testing, education, and controlled wallet workflows. This guide provides a self-contained Python script that generates a 32-byte private key, derives compressed and uncompressed public keys, produces legacy and native SegWit addresses, and prints a 24-word BIP39 mnemonic representation (direct entropy mapping) of the private key bytes.

• Keypair Generation: Creates a secp256k1 private key and derives both compressed and uncompressed public keys.
• Address Output: Prints P2PKH (legacy) addresses and a Bech32 P2WPKH (native SegWit) address.
• Mnemonic Support: Converts the 32-byte private key to a 24-word BIP39 mnemonic and verifies round-trip restoration.
• Network Selection: Supports mainnet and testnet prefixes for WIF and addresses.
• Cross-Platform Execution: Runs consistently on macOS, Windows, and Linux with standard Python tooling.
• Error Handling: Surfaces dependency and runtime errors with clear messages.
• Maintainability: Uses descriptive English comments and a simple CLI interface.

Important note: The 24-word mnemonic in this script represents the private key bytes as BIP39 entropy directly. For production HD-wallet workflows, the standard approach is typically BIP39 mnemonic → seed → BIP32/BIP84 derivation paths.

Python Script (v0.1.0):

Below is the complete script file content. All comments are written in English, and the header includes the requested attribution line.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# btc_keypair_mnemonic.py
# Version: 0.1.0
# © 2013–present Hyunsuk Frank Roh, MD — nGene Hemodynamic Research Center · All rights reserved
#
# Installation (Python 3.10+ recommended):
#   python3 -m venv .venv
#   source .venv/bin/activate  # macOS/Linux
#   .venv\Scripts\activate     # Windows PowerShell
#   pip install cryptography mnemonic
#
# Optional (only if RIPEMD160 is unavailable in hashlib on the platform):
#   pip install pycryptodome
#
# Usage examples:
#   python btc_keypair_mnemonic.py
#   python btc_keypair_mnemonic.py --testnet
#   python btc_keypair_mnemonic.py --language korean
#   python btc_keypair_mnemonic.py --mnemonic "word1 ... word24"
#
# Security notice:
#   Run on an offline, trusted machine. Never share private keys, WIF, or mnemonics.
#   No warranty; use at own risk.
#

from __future__ import annotations

import argparse
import hashlib
import sys
from dataclasses import dataclass
from typing import List, Optional

__version__ = "0.1.0"

# --- Dependency checks -------------------------------------------------------

try:
    from cryptography.hazmat.primitives.asymmetric import ec
except ImportError as exc:
    raise SystemExit(
        "Missing dependency: cryptography\n"
        "Install with: pip install cryptography"
    ) from exc

try:
    from mnemonic import Mnemonic
except ImportError as exc:
    raise SystemExit(
        "Missing dependency: mnemonic\n"
        "Install with: pip install mnemonic"
    ) from exc

# --- Constants ---------------------------------------------------------------

BASE58_ALPHABET = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"
BECH32_CHARSET = "qpzry9x8gf2tvdw0s3jn54khce6mua7l"

# --- Hash helpers ------------------------------------------------------------


def sha256(data: bytes) -> bytes:
    """Return SHA-256 digest."""

    return hashlib.sha256(data).digest()


def ripemd160(data: bytes) -> bytes:
    """
    Return RIPEMD-160 digest.

    hashlib supports RIPEMD-160 on many OpenSSL builds. If unavailable,
    pycryptodome can be used as a fallback.
    """

    try:
        h = hashlib.new("ripemd160")
        h.update(data)
        return h.digest()
    except ValueError:
        try:
            from Crypto.Hash import RIPEMD160  # type: ignore
        except ImportError as exc:
            raise RuntimeError(
                "RIPEMD160 is not available via hashlib on this platform.\n"
                "Install fallback with: pip install pycryptodome"
            ) from exc
        h = RIPEMD160.new()
        h.update(data)
        return h.digest()


def hash160(data: bytes) -> bytes:
    """Return HASH160 = RIPEMD160(SHA256(data))."""

    return ripemd160(sha256(data))


# --- Base58Check -------------------------------------------------------------


def base58_encode(raw: bytes) -> str:
    """Encode bytes into Base58 (no checksum)."""

    if not raw:
        return ""

    n = int.from_bytes(raw, "big")
    chars: List[str] = []

    while n > 0:
        n, r = divmod(n, 58)
        chars.append(BASE58_ALPHABET[r])

    # Preserve leading zero bytes as '1'
    pad = 0
    for b in raw:
        if b == 0:
            pad += 1
        else:
            break

    encoded = "".join(reversed(chars))
    return ("1" * pad) + encoded


def base58check_encode(payload: bytes) -> str:
    """Encode payload using Base58Check."""

    checksum = sha256(sha256(payload))[:4]
    return base58_encode(payload + checksum)


# --- Bech32 (BIP173) ---------------------------------------------------------


def bech32_polymod(values: List[int]) -> int:
    """Internal Bech32 checksum helper."""

    generator = [0x3B6A57B2, 0x26508E6D, 0x1EA119FA, 0x3D4233DD, 0x2A1462B3]
    chk = 1

    for v in values:
        top = chk >> 25
        chk = ((chk & 0x1FFFFFF) << 5) ^ v
        for i in range(5):
            chk ^= generator[i] if ((top >> i) & 1) else 0

    return chk


def bech32_hrp_expand(hrp: str) -> List[int]:
    """Expand HRP for checksum computation."""

    return [ord(x) >> 5 for x in hrp] + [0] + [ord(x) & 31 for x in hrp]


def bech32_create_checksum(hrp: str, data: List[int]) -> List[int]:
    """Create Bech32 checksum (BIP173, not bech32m)."""

    values = bech32_hrp_expand(hrp) + data
    polymod = bech32_polymod(values + [0, 0, 0, 0, 0, 0]) ^ 1
    return [(polymod >> (5 * (5 - i))) & 31 for i in range(6)]


def bech32_encode(hrp: str, data: List[int]) -> str:
    """Encode a Bech32 string."""

    combined = data + bech32_create_checksum(hrp, data)
    return hrp + "1" + "".join(BECH32_CHARSET[d] for d in combined)


def convertbits(data: bytes, frombits: int, tobits: int, pad: bool = True) -> List[int]:
    """
    General power-of-2 base conversion.

    Used for 8-bit byte arrays to 5-bit Bech32 words.
    """

    acc = 0
    bits = 0
    ret: List[int] = []
    maxv = (1 << tobits) - 1

    for value in data:
        if value < 0 or value >> frombits:
            raise ValueError("Invalid value for convertbits")
        acc = (acc << frombits) | value
        bits += frombits
        while bits >= tobits:
            bits -= tobits
            ret.append((acc >> bits) & maxv)

    if pad:
        if bits:
            ret.append((acc << (tobits - bits)) & maxv)
    else:
        if bits >= frombits:
            raise ValueError("Excess padding in convertbits")
        if (acc << (tobits - bits)) & maxv:
            raise ValueError("Non-zero padding in convertbits")

    return ret


# --- Bitcoin key/address utilities ------------------------------------------


def generate_private_key_bytes() -> bytes:
    """Generate a new secp256k1 private key (32 bytes)."""

    private_key = ec.generate_private_key(ec.SECP256K1())
    private_value = private_key.private_numbers().private_value
    return private_value.to_bytes(32, "big")


def private_key_bytes_to_public_key_bytes(privkey_bytes: bytes, compressed: bool = True) -> bytes:
    """Derive public key bytes from a 32-byte private key."""

    if len(privkey_bytes) != 32:
        raise ValueError("Private key must be 32 bytes")

    private_value = int.from_bytes(privkey_bytes, "big")
    private_key = ec.derive_private_key(private_value, ec.SECP256K1())
    public_numbers = private_key.public_key().public_numbers()
    x = public_numbers.x
    y = public_numbers.y

    x_bytes = x.to_bytes(32, "big")
    if not compressed:
        y_bytes = y.to_bytes(32, "big")
        return b"\x04" + x_bytes + y_bytes

    prefix = b"\x02" if (y % 2 == 0) else b"\x03"
    return prefix + x_bytes


def private_key_to_wif(privkey_bytes: bytes, compressed: bool = True, testnet: bool = False) -> str:
    """Convert private key bytes to WIF (Wallet Import Format)."""

    if len(privkey_bytes) != 32:
        raise ValueError("Private key must be 32 bytes")

    version = b"\xef" if testnet else b"\x80"
    payload = version + privkey_bytes + (b"\x01" if compressed else b"")
    return base58check_encode(payload)


def public_key_to_p2pkh_address(pubkey_bytes: bytes, testnet: bool = False) -> str:
    """Create a legacy P2PKH address from a public key."""

    version = b"\x6f" if testnet else b"\x00"
    payload = version + hash160(pubkey_bytes)
    return base58check_encode(payload)


def public_key_to_p2wpkh_address(pubkey_bytes: bytes, testnet: bool = False) -> str:
    """
    Create a native SegWit (P2WPKH) Bech32 address from a compressed public key.

    Note: P2WPKH requires a compressed public key.
    """

    if len(pubkey_bytes) != 33:
        raise ValueError("Compressed public key (33 bytes) is required for P2WPKH")

    hrp = "tb" if testnet else "bc"
    witness_version = 0
    program = hash160(pubkey_bytes)
    data = [witness_version] + convertbits(program, 8, 5, pad=True)
    return bech32_encode(hrp, data)


# --- BIP39 mnemonic conversion (entropy <-> words) --------------------------


def private_key_to_mnemonic(privkey_bytes: bytes, language: str = "english") -> str:
    """
    Convert a 32-byte private key to a 24-word BIP39 mnemonic.

    Important: This maps the private key bytes to BIP39 entropy directly.
    This is a representation technique, not a standard HD-wallet derivation flow.
    """

    if len(privkey_bytes) != 32:
        raise ValueError("Private key must be 32 bytes to produce a 24-word mnemonic")

    return Mnemonic(language).to_mnemonic(privkey_bytes)


def mnemonic_to_private_key(mnemonic: str, language: str = "english") -> bytes:
    """
    Convert a 24-word BIP39 mnemonic back into 32 bytes of entropy.

    This matches the direct-mapping approach used in private_key_to_mnemonic().
    """

    entropy = Mnemonic(language).to_entropy(mnemonic)
    if len(entropy) != 32:
        raise ValueError("Expected a 24-word mnemonic (32-byte entropy)")

    return entropy


# --- CLI ---------------------------------------------------------------------


@dataclass(frozen=True)
class OutputBundle:
    network: str
    private_key_hex: str
    wif_compressed: str
    wif_uncompressed: str
    public_key_compressed_hex: str
    public_key_uncompressed_hex: str
    p2pkh_compressed: str
    p2pkh_uncompressed: str
    p2wpkh_bech32: str
    mnemonic_24: str
    mnemonic_roundtrip_ok: bool


def build_bundle(privkey_bytes: bytes, language: str, testnet: bool) -> OutputBundle:
    pub_c = private_key_bytes_to_public_key_bytes(privkey_bytes, compressed=True)
    pub_u = private_key_bytes_to_public_key_bytes(privkey_bytes, compressed=False)

    wif_c = private_key_to_wif(privkey_bytes, compressed=True, testnet=testnet)
    wif_u = private_key_to_wif(privkey_bytes, compressed=False, testnet=testnet)

    addr_p2pkh_c = public_key_to_p2pkh_address(pub_c, testnet=testnet)
    addr_p2pkh_u = public_key_to_p2pkh_address(pub_u, testnet=testnet)

    addr_p2wpkh = public_key_to_p2wpkh_address(pub_c, testnet=testnet)

    mnemonic_24 = private_key_to_mnemonic(privkey_bytes, language=language)
    roundtrip = mnemonic_to_private_key(mnemonic_24, language=language) == privkey_bytes

    return OutputBundle(
        network="testnet" if testnet else "mainnet",
        private_key_hex=privkey_bytes.hex(),
        wif_compressed=wif_c,
        wif_uncompressed=wif_u,
        public_key_compressed_hex=pub_c.hex(),
        public_key_uncompressed_hex=pub_u.hex(),
        p2pkh_compressed=addr_p2pkh_c,
        p2pkh_uncompressed=addr_p2pkh_u,
        p2wpkh_bech32=addr_p2wpkh,
        mnemonic_24=mnemonic_24,
        mnemonic_roundtrip_ok=roundtrip,
    )


def print_bundle(bundle: OutputBundle) -> None:
    print(f"btc_keypair_mnemonic.py v{__version__}")
    print(f"Network: {bundle.network}")
    print("")
    print(f"Private key (hex): {bundle.private_key_hex}")
    print(f"Private key (WIF, compressed): {bundle.wif_compressed}")
    print(f"Private key (WIF, uncompressed): {bundle.wif_uncompressed}")
    print("")
    print(f"Public key (compressed, hex): {bundle.public_key_compressed_hex}")
    print(f"Public key (uncompressed, hex): {bundle.public_key_uncompressed_hex}")
    print("")
    print(f"Address (P2PKH, from compressed pubkey): {bundle.p2pkh_compressed}")
    print(f"Address (P2PKH, from uncompressed pubkey): {bundle.p2pkh_uncompressed}")
    print(f"Address (P2WPKH Bech32, native SegWit): {bundle.p2wpkh_bech32}")
    print("")
    print(f"Mnemonic (BIP39, 24 words): {bundle.mnemonic_24}")
    print(f"Mnemonic round-trip check: {'OK' if bundle.mnemonic_roundtrip_ok else 'FAIL'}")


def parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description=(
            "Generate a Bitcoin private key + address pair and print a 24-word BIP39 mnemonic "
            "for the private key bytes."
        )
    )
    parser.add_argument(
        "--testnet",
        action="store_true",
        help="Use Bitcoin testnet address/WIF prefixes.",
    )
    parser.add_argument(
        "--language",
        default="english",
        help="BIP39 wordlist language supported by the 'mnemonic' package (e.g., english, korean).",
    )
    parser.add_argument(
        "--mnemonic",
        default=None,
        help="Restore mode: provide an existing 24-word mnemonic to reconstruct the private key.",
    )
    return parser.parse_args(argv)


def main(argv: Optional[List[str]] = None) -> int:
    args = parse_args(argv)

    try:
        if args.mnemonic:
            privkey_bytes = mnemonic_to_private_key(args.mnemonic.strip(), language=args.language)
        else:
            privkey_bytes = generate_private_key_bytes()

        bundle = build_bundle(privkey_bytes, language=args.language, testnet=args.testnet)
        print_bundle(bundle)
        return 0

    except Exception as exc:
        print(f"Error: {exc}", file=sys.stderr)
        return 1


if __name__ == "__main__":
    raise SystemExit(main())

Written on December 21th, 2025

Embedded Scripting Interface

An Embedded Scripting Interface is a component within a software application that allows users to write and execute custom scripts in real-time using a programming language such as Python or R. This interface provides the flexibility to interact with the application’s internal functionality, perform complex computations, or extend features without modifying the core code. By integrating interpreters like Python or R, users can leverage powerful scripting capabilities, making the software highly customizable and adaptable to various use cases such as data analysis, automation, or integration with other tools.

R Packages

Installation of rpy2

If the rpy2 package is not yet available, it can be installed via pip using the following command:

pip install rpy2

Utilizing R within Python using rpy2

Once the rpy2 package is installed, R packages and functions can be accessed from Python. The following example demonstrates how to achieve this:

# Import rpy2's interface to R
import rpy2.robjects as ro
from rpy2.robjects.packages import importr
from rpy2.robjects.vectors import FloatVector

# Import the base R package
base = importr('base')

# Example of using a simple R function, such as the sum function
r_sum = ro.r['sum']  # Access R's sum function
result = r_sum(FloatVector([1, 2, 3, 4, 5]))

print(f"Sum calculated using R: {result[0]}")

# Example of installing and using a specific R package
utils = importr('utils')
utils.install_packages('ggplot2')  # Install an R package like ggplot2  

Executing R Scripts from Python

In instances where a more complex R script is required, the subprocess module can be utilized to execute entire R scripts from within Python:

import subprocess

# Running an R script from Python
subprocess.run(['Rscript', 'your_script.R'])  

This approach facilitates seamless interaction with R functions and packages within a Python environment, enabling more versatile workflows that combine the strengths of both languages.

Python Interpreter

To enable interaction with a Python programming interface within standalone software, an embedded Python interpreter can be integrated. This allows the execution of Python code within the application's environment. Several approaches can be employed based on the requirements:

1. Using the code Module (Interactive Console):

Python provides a built-in module called code that can be used to embed an interactive interpreter session within an application. This method is suitable for both GUI and CLI applications. For instance, by invoking the InteractiveConsole class, users can input Python code, which is executed in real-time within the program's environment. This method facilitates straightforward access to Python's functionality.

import code

# Open an interactive console for user input
def start_interpreter():
   console = code.InteractiveConsole(locals=globals())
   console.interact("Welcome to the Python Interactive Console within your software.")

start_interpreter()

2. Using exec() for Inline Code Execution:

Another option is to allow dynamic execution of Python code using the exec() function. This method enables users to input code as a string, which is then executed in the program's context. It is a flexible and dynamic approach for integrating Python scripting capabilities.

def execute_user_code():
   user_code = input("Enter Python code: ")
   try:
      exec(user_code, globals())  # Execute in the global context
   except Exception as e:
      print(f"Error executing code: {e}")

execute_user_code()

3. Building a Full IDE or Editor:

For a more sophisticated experience, a complete IDE or code editor can be embedded using GUI frameworks like Tkinter or PyQt/PySide. These frameworks allow the creation of text editors that provide a user-friendly interface for writing and executing Python code.

import tkinter as tk
from tkinter import scrolledtext

def execute_code():
   code_to_run = editor.get("1.0", tk.END)
   try:
      exec(code_to_run)
   except Exception as e:
      output_box.insert(tk.END, f"Error: {e}\n")

root = tk.Tk()
root.title("Python IDE")

editor = scrolledtext.ScrolledText(root, wrap=tk.WORD, width=60, height=20)
editor.pack()

run_button = tk.Button(root, text="Run Code", command=execute_code)
run_button.pack()

output_box = scrolledtext.ScrolledText(root, wrap=tk.WORD, width=60, height=10)
output_box.pack()

root.mainloop()

4. Embedding IPython or Jupyter Kernel:

For a more powerful interactive environment, IPython or a Jupyter kernel can be embedded. IPython offers an enhanced interactive shell that can be integrated into the application. Additionally, running a Jupyter kernel in the background allows for a richer interface, which can connect to the application for advanced coding and data manipulation.

from IPython import embed

# Start an IPython session within the software
embed()

Installing `rpy2` in PyCharm on macOS

This guide provides detailed steps for installing and configuring the `rpy2` Python package within PyCharm on macOS, with special consideration given to different R installation methods. For installations where R has been installed via a `.pkg` file rather than Homebrew, specific path configurations may be required. Additionally, guidance is included for installing R via Homebrew, which may streamline the setup process for `rpy2`.

Step 1: Activate the Virtual Environment in PyCharm

1. Open the Terminal application on macOS.
2. Activate the project-specific virtual environment by executing the following command:

   source /Users/frank/PycharmProjects/tmpPy/.venv/bin/activate

   This step establishes the correct environment for the `rpy2` installation within PyCharm.

Step 2: Execute the Corrected Command for Installing `rpy2`

Given that PyCharm’s installation path includes a space in “PyCharm CE,” an escape character (`\`) is required to prevent command errors. To install `rpy2` using PyCharm’s packaging tool:

1. In the terminal, enter the following command:

   /Users/frank/PycharmProjects/tmpPy/.venv/bin/python /Applications/PyCharm\ CE.app/Contents/plugins/python-ce/helpers/packaging_tool.py install rpy2

   This command directly installs `rpy2` into the virtual environment by utilizing PyCharm’s internal package installer.

Step 3: Verify the Installation in PyCharm

1. In PyCharm, navigate to View > Tool Windows > Python Packages.
2. Search for `rpy2` in the available packages list to confirm its presence within the environment.

Additional Configuration for R Installed via .pkg Installer

If R has been installed using a `.pkg` file rather than Homebrew, the system may not automatically recognize the R installation path. To address this:

Determine the R Installation Path

1. To locate the R installation path, enter the following command in the terminal:

   which R

   This command outputs the path to the R executable, typically /Library/Frameworks/R.framework/Resources for `.pkg` installations.

Set the R_HOME Environment Variable

1. Set the R_HOME environment variable to the installation path by adding the following line to your shell profile file (~/.zshrc for Zsh or ~/.bash_profile for Bash) to make it persistent:

   export R_HOME=/Library/Frameworks/R.framework/Resources

2. After adding this line, reload the terminal profile with:

   source ~/.zshrc  
   # or
   source ~/.bash_profile  

This configuration ensures `rpy2` can locate the R libraries when using an R version installed via `.pkg`.

Optional: Installing R Using Homebrew

For those who prefer to install R via Homebrew, which may simplify `rpy2` configuration:

1. Install Homebrew if it is not already installed:

   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

2. Install R with Homebrew:

   brew install r

   This command installs R in a standard Homebrew location, typically /usr/local/Cellar/r/<version>, and sets up necessary environment variables automatically.
3. Set R_HOME (if required):
   • After installation, the R path can be found by running:

     brew --prefix r

   • If additional configuration is needed, add the output path from the above command as the R_HOME value in the shell profile.

Using R Packages in Python with rpy2

A. Loading R Functions from a String or File

Example: R Function Definition

Suppose you have an R function named myFunc that processes some data:

myFunc <- function(x) {
  return(x * 2)
}

Using SignatureTranslatedAnonymousPackage

1. Creating an R package from an R code string:

from rpy2.robjects.packages import SignatureTranslatedAnonymousPackage
from rpy2 import robjects

# R code as a string
r_code = """
myFunc <- function(x) {
    return(x * 2)
}
"""

# Create a temporary R package from the string
my_func_package = SignatureTranslatedAnonymousPackage(r_code, "myFuncPackage")

2. Using the loaded R function:

# Call the R function from Python
result = my_func_package.myFunc(10)  # Pass data to R function
print("The result is:", result[0])  # Convert the R result to Python type for display

Here, myFunc is an attribute of my_func_package, allowing you to invoke the R function with Python data.

Using SignatureTranslatedPackage

If you have an R script in a file, for example myRFunctions.R, containing your R functions, you can load it as:

from rpy2.robjects.packages import SignatureTranslatedAnonymousPackage

with open("myRFunctions.R", "r") as r_file:
    r_code_from_file = r_file.read()

# Create an R package from the file's content
my_functions_package = SignatureTranslatedAnonymousPackage(r_code_from_file, "myFunctionsPackage")

Then call the functions from this package as shown above.

Data Conversion

• rpy2 automatically converts basic data types between Python and R:
• Python lists and numpy arrays to R vectors.
• Python integers and floats to R numeric vectors.
• If further conversion is necessary, rpy2 offers methods like numpy2ri.activate() to handle numpy arrays. Ensure appropriate conversions if dealing with complex data structures.

B. Using R Functions from External Packages

Step 1: Import R Packages

import rpy2.robjects.packages as rpackages

# Import R's "utils" package to install packages
utils = rpackages.importr('utils')
utils.chooseCRANmirror(ind=1)  # Select a CRAN mirror

# Install a needed R package from CRAN if not already installed
package_name = 'fastICA'
if not rpackages.isinstalled(package_name):
    utils.install_packages(package_name)

Step 2: Import the R Package as a Python Object

fastica_pkg = rpackages.importr('fastICA')

Step 3: Use Functions from the Imported Package

# Example: Using the fastICA function from the fastICA package
import numpy as np

# Prepare input data (converted to R-compatible format if needed)
data = np.random.rand(100, 2)  # 100 observations, 2 variables

# Convert numpy array to R matrix
data_r = robjects.r.matrix(data, nrow=data.shape[0], ncol=data.shape[1])

# Call the fastICA function from the fastICA R package
ica_result = fastica_pkg.fastICA(data_r, n_comp=2)

This approach allows you to use any R package and its functions similarly.

C. Using R Scripts and Packages Generally

1. Loading an Entire R Script

If you have an R script, myRScript.R, containing multiple function definitions or code lines:

with open("myRScript.R", "r") as file:
    r_script = file.read()

# Convert the R script into a Python-callable form
my_r_package = SignatureTranslatedAnonymousPackage(r_script, "myRPackage")

# Now you can call the R functions defined in 'myRScript.R'
result = my_r_package.someFunction(some_args)

Ensure that someFunction is defined in myRScript.R.

2. Handling Data Exchange

• You can pass Python lists or numpy arrays to R functions by converting them to R vectors or matrices. Using numpy2ri can simplify this conversion:

from rpy2.robjects import numpy2ri
    numpy2ri.activate()    

• Data returned from R functions can also be converted back to Python using automatic conversion or by specific methods (e.g., .rx2(), .rx(), or directly indexing the returned object if it’s a known structure).

D. Another Example: The Provided fastICA R Script

Using the exact steps above:

1. Load the R script: Wrap the R code for fastICA into a string in Python or read it from a .R file.
2. Create the R package: Using SignatureTranslatedAnonymousPackage.
3. Use the function: Once loaded, the fastICA function becomes accessible through the created package object, where you can pass data arrays from Python (converted to R matrices) and retrieve results.

Integrating R's fastICA with Python using rpy2 on Windows

The error encountered:

ValueError: The system "%s" is not supported.

indicates that rpy2 is unable to recognize the operating system correctly. This issue often arises when rpy2 cannot locate the R installation on a Windows system or when there is a mismatch between the architectures of Python and R (e.g., 64-bit vs. 32-bit).

To resolve this issue and successfully integrate R's fastICA into a Python script using rpy2 on Windows, the following detailed steps should be followed:

1. Ensure R is Installed on the Windows System

Before using rpy2, it is necessary to have R installed on the machine.

Steps:

1. Download R:

   • Visit the CRAN R for Windows page.
   • Download the latest version of R (e.g., R-4.3.0-win.exe).

2. Install R:

   • Run the downloaded installer.
   • Follow the installation prompts, accepting default settings unless specific configurations are required.
   • Note the installation directory (commonly C:\Program Files\R\R-4.3.0).

2. Set Environment Variables for R

rpy2 relies on environment variables to locate the R installation. Setting R_HOME and updating the PATH variable is essential.

Steps:

1. Find R Installation Path:

   • Typically, R is installed in C:\Program Files\R\R-x.x.x (e.g., C:\Program Files\R\R-4.3.0).

2. Set R_HOME Environment Variable:

   • Access Environment Variables:

     • Right-click on This PC or My Computer on the desktop or in File Explorer.
     • Select Properties.
     • Click on Advanced system settings.
     • In the System Properties window, click on the Environment Variables button.

   • Create R_HOME:

     • Under System variables, click New.
     • Set Variable name to R_HOME.
     • Set Variable value to the path of the R installation (e.g., C:\Program Files\R\R-4.3.0).
     • Click OK.

3. Add R's bin Directory to PATH:

   • Still in the Environment Variables window:
   • Under System variables, find and select the Path variable, then click Edit.
   • Click New and add the path to R's bin directory (e.g., C:\Program Files\R\R-4.3.0\bin).
   • Click OK to save changes.

4. Verify Environment Variables:

   • Open Command Prompt and run:

   echo %R_HOME%

   This should display the path to the R installation.

   • Test if R is accessible by running:

   R --version

   This should display the R version information.

3. Ensure Python and R Architectures Match

It is crucial that both Python and R are either 64-bit or 32-bit. Mismatched architectures can lead to compatibility issues.

Steps:

1. Check Python Architecture:

   • Open Python interpreter and run:

   import platform
   print(platform.architecture())

   The output should be similar to ('64bit', 'WindowsPE').

2. Check R Architecture:

   • Open R GUI or RStudio and run:

   .Machine$sizeof.pointer

   The output should be 8 for 64-bit or 4 for 32-bit.

3. Reinstall if Necessary:

   • If there is a mismatch, reinstall Python or R to match architectures.

4. Install or Upgrade rpy2

Ensuring that rpy2 is installed and compatible with the R version is necessary.

Steps:

1. Upgrade pip:

   pip install --upgrade pip

2. Install rpy2:

   pip install --upgrade rpy2

   Note: Ensure that rpy2 is installed in the same Python environment used for the project (e.g., a virtual environment).

3. Verify rpy2 Installation:

   In Python, run:

   import rpy2.robjects as robjects
   print(robjects.r('version'))

   This should display R's version information without errors.

Troubleshooting:

If errors are encountered, ensure that R is correctly installed and that the environment variables are properly set.

5. Test rpy2 Integration with R

Before integrating the fastICA function, verifying that rpy2 can interact with R is essential.

Steps:

1. Simple Test:

   Create a Python script (e.g., test_rpy2.py) with the following content:

   import rpy2.robjects as robjects
   
   # Print R version
   print(robjects.r('version'))
   
   # Execute a simple R command
   r_sum = robjects.r('sum(c(1, 2, 3, 4, 5))')
   print("Sum from R:", r_sum[0])

2. Run the Script:

   python test_rpy2.py

3. Expected Output:

   • R version details.
   • Sum from R: 15

4. If Errors Occur:

   Revisit the previous steps to ensure R is correctly installed and environment variables are set.

6. Integrate fastICA R Function into Python

Proceed to integrate the fastICA R function into the Python script.

Complete Python Script with R fastICA Integration:

import os
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import wavfile
from rpy2 import robjects
from rpy2.robjects import numpy2ri
from rpy2.robjects.packages import SignatureTranslatedAnonymousPackage

# Enable the conversion between numpy arrays and R vectors/matrices
numpy2ri.activate()

# Set the maximum vector size in R to attempt to prevent memory limit issues
os.environ['R_MAX_VSIZE'] = '32GB'  # Adjust based on available system memory

# Load the fastICA.R script content
fastICA_r_script = """
fastICA <-
function (X, n.comp, alg.typ = c("parallel","deflation"),
          fun = c("logcosh", "exp"),
          alpha = 1, method = c("R", "C"),
          row.norm = FALSE, maxit = 200, tol = 1e-04,
          verbose = FALSE, w.init=NULL)
{
    dd <- dim(X)
    d <- dd[dd != 1L]
    if (length(d) != 2L)
        stop("data must be matrix-conformal")
    X <- if (length(d) != length(dd)) matrix(X, d[1L], d[2L])
    else as.matrix(X)

    if (alpha < 1 || alpha > 2)
        stop("alpha must be in range [1,2]")
    method <- match.arg(method)
    alg.typ <- match.arg(alg.typ)
    fun <- match.arg(fun)
    n <- nrow(X)
    p <- ncol(X)

    if (n.comp > min(n, p)) {
        message("'n.comp' is too large: reset to ", min(n, p))
        n.comp <- min(n, p)
    }
    if(is.null(w.init))
        w.init <- matrix(rnorm(n.comp^2),n.comp,n.comp)
    else {
        if(!is.matrix(w.init) || length(w.init) != (n.comp^2))
            stop("w.init is not a matrix or is the wrong size")
    }
    if (method == "R") {
        if (verbose) message("Centering")

        X <- scale(X, scale = FALSE)

        X <- if (row.norm) t(scale(X, scale=row.norm)) else t(X)

        if (verbose) message("Whitening")
        V <- X %*% t(X)/n

        s <- La.svd(V)
        D <- diag(c(1/sqrt(s$d)))

        K <- D %*% t(s$u)
        K <- matrix(K[1:n.comp, ], n.comp, p)
        X1 <- K %*% X
        a <- if (alg.typ == "deflation")
            ica.R.def(X1, n.comp, tol = tol, fun = fun,
                      alpha = alpha, maxit = maxit, verbose = verbose, w.init = w.init)
        else if (alg.typ == "parallel")
            ica.R.par(X1, n.comp, tol = tol, fun = fun,
                      alpha = alpha, maxit = maxit, verbose = verbose, w.init = w.init)
        w <- a %*% K
        S <- w %*% X
        A <- t(w) %*% solve(w %*% t(w))
        return(list(X = t(X), K = t(K), W = t(a), A = t(A), S = t(S)))
    } else if (method == "C") {
        a <- .C(icainc_JM,
                as.double(X),
                as.double(w.init),
                as.integer(p),
                as.integer(n),
                as.integer(n.comp),
                as.double(alpha),
                as.integer(1),
                as.integer(row.norm),
                as.integer(1L + (fun == "exp")),
                as.integer(maxit),
                as.double(tol),
                as.integer(alg.typ != "parallel"),
                as.integer(verbose),
                X = double(p * n),
                K = double(n.comp * p),
                W = double(n.comp * n.comp),
                A = double(p * n.comp),
                S = double(n.comp * n))
        X1 <- matrix(a$X, n, p)
        K <- matrix(a$K, p, n.comp)
        W <- matrix(a$W, n.comp, n.comp)
        A <- matrix(a$A, n.comp, p)
        S <- matrix(a$S, n, n.comp)
        list(X = X1, K = K, W = W, A = A, S = S)
    }
}

ica.R.def <-
    function (X, n.comp, tol, fun, alpha, maxit, verbose, w.init)
{
    if (verbose && fun == "logcosh")
        message("Deflation FastICA using logcosh approx. to neg-entropy function")
    if (verbose && fun =="exp")
        message("Deflation FastICA using exponential approx. to neg-entropy function")
    p <- ncol(X)
    W <- matrix(0, n.comp, n.comp)
    for (i in 1:n.comp) {
        if (verbose) message("Component ", i)
        w <- matrix(w.init[i,], n.comp, 1)
        if (i > 1) {
            t <- w
            t[1:length(t)] <- 0
            for (u in 1:(i - 1)) {
                k <- sum(w * W[u, ])
                t <- t + k * W[u, ]
            }
            w <- w - t
        }
        w <- w/sqrt(sum(w^2))
        lim <- rep(1000, maxit)
        it <- 1
        if (fun == "logcosh") {
            while (lim[it] > tol && it < maxit) {
                wx <- t(w) %*% X
                gwx <- tanh(alpha * wx)
                gwx <- matrix(gwx, n.comp, p, byrow = TRUE)
                xgwx <- X * gwx
                v1 <- apply(xgwx, 1, FUN = mean)
                g.wx <- alpha * (1 - (tanh(alpha * wx))^2)
                v2 <- mean(g.wx) * w
                w1 <- v1 - v2
                w1 <- matrix(w1, n.comp, 1)
                it <- it + 1
                if (i > 1) {
                    t <- w1
                    t[1:length(t)] <- 0
                    for (u in 1:(i - 1)) {
                        k <- sum(w1 * W[u, ])
                        t <- t + k * W[u, ]
                    }
                    w1 <- w1 - t
                }
                w1 <- w1/sqrt(sum(w1^2))
                lim[it] <- Mod(Mod(sum((w1 * w))) - 1)
                if (verbose)
                    message("Iteration ", it - 1, " tol = ", format(lim[it]))
                w <- matrix(w1, n.comp, 1)
            }
        }
        if (fun == "exp") {
            while (lim[it] > tol && it < maxit) {
                wx <- t(w) %*% X
                gwx <- wx * exp(-(wx^2)/2)
                gwx <- matrix(gwx, n.comp, p, byrow = TRUE)
                xgwx <- X * gwx
                v1 <- apply(xgwx, 1, FUN = mean)
                g.wx <- (1 - wx^2) * exp(-(wx^2)/2)
                v2 <- mean(g.wx) * w
                w1 <- v1 - v2
                w1 <- matrix(w1, n.comp, 1)
                it <- it + 1
                if (i > 1) {
                    t <- w1
                    t[1:length(t)] <- 0
                    for (u in 1:(i - 1)) {
                        k <- sum(w1 * W[u, ])
                        t <- t + k * W[u, ]
                    }
                    w1 <- w1 - t
                }
                w1 <- w1/sqrt(sum(w1^2))
                lim[it] <- Mod(Mod(sum((w1 * w))) - 1)
                if (verbose)
                    message("Iteration ", it - 1, " tol = ", format(lim[it]))
                w <- matrix(w1, n.comp, 1)
            }
        }
        W[i, ] <- w
    }
    W
}

ica.R.par <- function (X, n.comp, tol, fun, alpha, maxit, verbose, w.init)
{
    Diag <- function(d) if(length(d) > 1L) diag(d) else as.matrix(d)
    p <- ncol(X)
    W <- w.init
    sW <- La.svd(W)
    W <- sW$u %*% Diag(1/sW$d) %*% t(sW$u) %*% W
    W1 <- W
    lim <- rep(1000, maxit)
    it <- 1
    if (fun == "logcosh") {
        if (verbose)
            message("Symmetric FastICA using logcosh approx. to neg-entropy function")
        while (lim[it] > tol && it < maxit) {
            wx <- W %*% X
            gwx <- tanh(alpha * wx)
            v1 <- gwx %*% t(X)/p
            g.wx <- alpha * (1 - (gwx)^2)
            v2 <- Diag(apply(g.wx, 1, FUN = mean)) %*% W
            W1 <- v1 - v2
            sW1 <- La.svd(W1)
            W1 <- sW1$u %*% Diag(1/sW1$d) %*% t(sW1$u) %*% W1
            lim[it + 1] <- max(Mod(Mod(diag(W1 %*% t(W))) - 1))
            W <- W1
            if (verbose)
                message("Iteration ", it, " tol = ", format(lim[it + 1]))
            it <- it + 1
        }
    }
    if (fun == "exp") {
        if (verbose)
            message("Symmetric FastICA using exponential approx. to neg-entropy function")
        while (lim[it] > tol && it < maxit) {
            wx <- W %*% X
            gwx <- wx * exp(-(wx^2)/2)
            v1 <- gwx %*% t(X)/p
            g.wx <- (1 - wx^2) * exp(-(wx^2)/2)
            v2 <- Diag(apply(g.wx, 1, FUN = mean)) %*% W
            W1 <- v1 - v2
            sW1 <- La.svd(W1)
            W1 <- sW1$u %*% Diag(1/sW1$d) %*% t(sW1$u) %*% W1
            lim[it + 1] <- max(Mod(Mod(diag(W1 %*% t(W))) - 1))
            W <- W1
            if (verbose)
                message("Iteration ", it, " tol = ", format(lim[it + 1]))
            it <- it + 1
        }
    }
    W
}
"""

# Create an R package with the fastICA function
fastICA_r = SignatureTranslatedAnonymousPackage(fastICA_r_script, "fastICA")

# Define function to read and normalize .wav audio files using only the first few seconds
def fetch_wav_and_convert_to_numpy(file_path, duration_seconds=1):
    sample_rate, audio_data = wavfile.read(file_path)
    max_samples = int(sample_rate * duration_seconds)
    audio_data = audio_data[:max_samples]  # Use only the first few seconds of the data
    # Normalize if not silent audio
    if np.max(np.abs(audio_data)) != 0:
        normalized_audio = audio_data / np.max(np.abs(audio_data))
    else:
        normalized_audio = audio_data
    return normalized_audio, sample_rate

# Function to plot waveforms using matplotlib
def plot_waveforms(signals, sample_rate, titles, colors):
    total_num_plots = len(signals)
    plt.figure(figsize=(6.5, 3 * total_num_plots), dpi=100)
    for i, (audio_array, title, color) in enumerate(zip(signals, titles, colors), start=1):
        time_axis = np.linspace(0, len(audio_array) / sample_rate, num=len(audio_array))
        plt.subplot(total_num_plots, 1, i)
        plt.plot(time_axis, audio_array, label=title, color=color)
        plt.title(title)
        plt.xlabel("Time (seconds)")
        plt.ylabel("Amplitude (normalized)")
        plt.legend()
        plt.tight_layout()
    plt.show()

# Main function to perform ICA using fastICA in R
def run_ica():
    heart_sound_path = "heart_sound.wav"
    lung_sound_path = "lung_sound.wav"

    # Fetch the first few seconds of audio for memory efficiency
    heart_sound, sr_heart = fetch_wav_and_convert_to_numpy(heart_sound_path, duration_seconds=1)
    lung_sound, sr_lung = fetch_wav_and_convert_to_numpy(lung_sound_path, duration_seconds=1)

    # Ensure both audio files have the same sample rate
    if sr_heart != sr_lung:
        raise ValueError("Sample rates do not match.")

    sr = sr_heart  # Using the common sample rate

    # Stack both signals for ICA; shape will be (2, number_of_samples)
    S = np.vstack((heart_sound, lung_sound))

    # Transpose the data to match R's expected input format (observations × variables)
    # fastICA expects data rows as observations and columns as variables/features
    S_transposed = S.T

    # Convert the numpy array to an R matrix
    S_r = robjects.r.matrix(S_transposed, nrow=S_transposed.shape[0], ncol=S_transposed.shape[1])

    # Perform ICA using the R function
    # Use underscores in argument names for Python function calls
    result = fastICA_r.fastICA(
        S_r,
        n_comp=2,  # Fixed: replaced n.comp with n_comp
        alg_typ="parallel",
        fun="logcosh",
        maxit=300,
        tol=1e-04,
        verbose=True
    )

    # Extract components from R result
    components_r = np.array(result.rx2("S"))

    # The independent components matrix S from the R result is oriented as (observations × components)
    # Transpose it back to match the shape (components × samples) for plotting
    components = components_r.T

    # Prepare data for plotting
    signals = [heart_sound, lung_sound, components[:, 0], components[:, 1]]
    titles = [
        "Original Heart Sound Waveform",
        "Original Lung Sound Waveform",
        "Separated Signal 1",
        "Separated Signal 2"
    ]
    colors = ['red', 'blue', 'purple', 'cyan']

    # Plot the waveforms
    plot_waveforms(signals, sr, titles, colors)
    print("Waveforms have been plotted successfully.\n")

# Run the ICA process
if __name__ == "__main__":
    run_ica()

Explanation:

A. Loading R Functions into Python

• SignatureTranslatedAnonymousPackage: This function allows the creation of a temporary R package from a string containing R code. It translates R function signatures to be callable from Python.
• Handling R Scripts: The entire R script (fastICA_r_script) is embedded as a multi-line string in Python. It is essential to ensure that the R code is syntactically correct and free of errors.

B. Data Preparation for fastICA

• Reading .wav Files: The fetch_wav_and_convert_to_numpy function reads .wav files and normalizes them. Only the first few seconds (e.g., 1 second) are read to minimize memory usage.
• Data Orientation: R's fastICA expects data in the format (observations × variables). Python's numpy typically uses (variables × observations). Transposing the data (S_transposed = S.T) ensures compatibility.

C. Executing the R fastICA Function

• Function Parameters:
  • n.comp: Number of components to extract (e.g., 2).
  • alg.typ: Algorithm type ("parallel" or "deflation").
  • fun: Non-linear function ("logcosh" or "exp").
  • maxit: Maximum iterations (e.g., 300).
  • tol: Tolerance for convergence (e.g., 1e-04).
  • verbose: Enable verbose output (True for detailed logs).
• Handling Results: The fastICA function returns a list containing separated signals (S). Extract S using result.rx2("S") and transpose it for plotting.

D. Plotting the Results

• Visualization: Matplotlib is used to plot the original and separated signals. Each signal is plotted in a separate subplot for clarity.

7. Detailed Explanation and Best Practices

A. Loading R Functions into Python

• SignatureTranslatedAnonymousPackage: This function facilitates the creation of a temporary R package from a string containing R code, enabling the translation of R function signatures to be callable from Python.
• Handling R Scripts: Embedding the entire R script as a multi-line string ensures that the R code is integrated seamlessly. It is imperative to verify that the R code is free from syntax errors.

B. Data Preparation for fastICA

• Reading .wav Files: The function reads and normalizes .wav files, focusing on the initial few seconds to optimize memory usage.
• Data Orientation: Transposing the data aligns it with R's expectations, facilitating accurate processing by the fastICA function.

C. Executing the R fastICA Function

• Function Parameters: Parameters are meticulously matched with the R-based fastICA usage to ensure consistency and accuracy in the analysis.
• Handling Results: Extracting and transposing the results appropriately prepares the data for subsequent visualization and analysis.

D. Plotting the Results

• Visualization: Utilizing matplotlib for plotting provides a clear and concise representation of both original and separated signals, enhancing interpretability.

8. Troubleshooting Common Issues

A. rpy2 Cannot Find R Installation

• Ensure R is Installed: Verify that R is correctly installed by running R --version in the Command Prompt.
• Check Environment Variables: Confirm that R_HOME points to the correct R installation directory and that R's bin directory is included in the PATH.
• Architecture Mismatch: Both Python and R should be either 64-bit or 32-bit. Mismatches can cause failures.

B. Permissions Issues

• Run as Administrator: Lack of permissions can prevent rpy2 from accessing R libraries. Running the Integrated Development Environment (e.g., PyCharm) as an administrator may resolve such issues.

C. R Package Dependencies

• Install Necessary R Packages: If the R script relies on additional packages (e.g., stats, base), ensure they are installed. R packages can be installed within the Python script using robjects.r:

  robjects.r('install.packages("packageName")')

D. Memory Limitations Persist

• Further Reduce Data Size: If memory issues continue, consider processing even shorter durations or downsampling the audio data.
• Optimize R Code: Ensure that the R script is optimized for memory usage by avoiding unnecessary data duplication.

9. Additional Tips

A. Using RStudio for Testing

Before integrating with Python, testing the fastICA.R script in RStudio ensures that it functions as expected with sample data.

B. Logging and Debugging

• Verbose Mode: Setting verbose=True in the fastICA function provides detailed logs of the ICA process.
• Catch Exceptions: Wrapping the ICA execution code in try-except blocks facilitates graceful error handling and debugging:

  try:
      result = fastICA_r.fastICA(
          S_r,
          n_comp=2,
          alg_typ="parallel",
          fun="logcosh",
          maxit=300,
          tol=1e-04,
          verbose=True
      )
  except Exception as e:
      print("An error occurred during ICA execution:", e)

C. Validate Separated Signals

• Listen to Separated Signals: Saving the separated signals as .wav files allows for auditory validation of the separation quality:

  from scipy.io.wavfile import write
  
  # Save separated signals
  write("separated_signal_1.wav", sr, (components[:, 0] * 32767).astype(np.int16))
  write("separated_signal_2.wav", sr, (components[:, 1] * 32767).astype(np.int16))

• Ensure Proper Normalization: Before saving, ensure that the separated signals are appropriately normalized to prevent clipping.

10. Final Thoughts

Integrating R functions into Python using rpy2 can be powerful but requires careful setup, especially on Windows systems. By following the steps outlined above, the ValueError should be resolved, and Independent Component Analysis (ICA) using R's fastICA within Python projects can be successfully performed.

If issues persist after following these steps, the following actions are recommended:

• Check rpy2 Documentation: Refer to the rpy2 official documentation for more detailed guidance and advanced configurations.
• Community Support: Seek assistance from forums such as Stack Overflow or the rpy2 GitHub repository for specific issues.
• Alternative Approaches: If integrating R proves too cumbersome, consider using Python's native ICA implementations (e.g., sklearn.decomposition.FastICA), which may offer easier integration and better performance within Python environments.

Troubleshooting rpy2 Script Execution in PyCharm on Windows

It is acknowledged that running the test_rpy2.py script via the terminal successfully yields the expected output:

R version details.
Sum from R: 15

However, encountering issues when executing the same script within PyCharm on a Windows system can be attributed to several factors. The following steps provide a comprehensive approach to diagnosing and resolving the problem:

1. Verify Python Interpreter Configuration in PyCharm

Steps:

1. Open Project Settings:
   • Navigate to File > Settings (or PyCharm > Preferences on macOS).
2. Check Python Interpreter:
   • In the Settings window, select Project: [Your Project Name] > Python Interpreter.
   • Ensure that the interpreter listed matches the one used in the terminal where the script runs successfully.
   • If multiple interpreters are available, select the appropriate one or add a new interpreter if necessary.
3. Apply Changes:
   • After selecting the correct interpreter, click Apply and then OK to save the changes.

2. Ensure Environment Variables Are Recognized by PyCharm

PyCharm may not inherit the environment variables set in the system by default. Specifically, R_HOME and the PATH variable pointing to R's bin directory must be accessible within PyCharm.

Steps:

1. Access Run Configurations:
   • Go to Run > Edit Configurations.
2. Select Your Script Configuration:
   • Choose the configuration for test_rpy2.py or create a new one if it doesn't exist.
3. Set Environment Variables:
   • In the selected configuration, locate the Environment variables section.
   • Click the ... button to open the environment variables dialog.
   • Add the following variables if they are not already present:
     • Variable Name: R_HOME
     • Variable Value: C:\Program Files\R\R-4.3.0 (replace with the actual R installation path)
   • Ensure that the R bin directory is included in the PATH variable. If not, append it:
     • Variable Name: PATH
     • Variable Value: %PATH%;C:\Program Files\R\R-4.3.0\bin (adjust the path as necessary)
4. Apply and Save:
   • Click OK to close the environment variables dialog.
   • Click Apply and then OK to save the run configuration.

3. Confirm rpy2 Installation in the Selected Interpreter

Ensure that rpy2 is installed in the Python interpreter that PyCharm is utilizing.

Steps:

1. Open Python Interpreter Settings:
   • Navigate to File > Settings > Project: [Your Project Name] > Python Interpreter.
2. Check Installed Packages:
   • In the list of installed packages, verify that rpy2 is present.
   • If rpy2 is missing, install it by clicking the + button, searching for rpy2, and proceeding with the installation.

4. Test R Accessibility Within PyCharm

Create a simple test within PyCharm to verify that R is accessible through rpy2.

Steps:

1. Create a New Python Script:
   • In the project, create a new Python file named pycharm_r_test.py.
2. Add the Following Code:

   import rpy2.robjects as robjects
   
   # Print R version
   print(robjects.r('version'))
   
   # Execute a simple R command
   r_sum = robjects.r('sum(c(1, 2, 3, 4, 5))')
   print("Sum from R:", r_sum[0])

3. Run the Script:
   • Right-click on pycharm_r_test.py and select Run 'pycharm_r_test'.

Expected Output:

R version details.
Sum from R: 15

If Errors Occur:

Note the error messages displayed in the Run window. Common issues may relate to environment variables, interpreter mismatches, or rpy2 installation problems.

5. Address Common Issues

A. Environment Variables Not Set Correctly

Revisit Step 2 to ensure that R_HOME and PATH are correctly configured within PyCharm's run configurations.

B. Interpreter Mismatch

Confirm that the Python interpreter used in PyCharm matches the one in the terminal where the script executes successfully.

C. Permissions Issues

Run PyCharm as an administrator to rule out permission-related problems:

• Right-click on the PyCharm shortcut and select Run as administrator.

D. Missing R Packages

Ensure that all necessary R packages required by rpy2 are installed. Install missing packages using R or within the Python script:

robjects.r('install.packages("packageName")')

6. Additional Troubleshooting Steps

A. Reinstall rpy2

Sometimes, reinstalling rpy2 can resolve underlying issues.

pip uninstall rpy2
pip install rpy2

B. Verify R Installation Path

Double-check the R installation path specified in R_HOME and PATH. Ensure there are no typos or incorrect directory references.

C. Check for Multiple R Installations

Having multiple R versions installed can cause conflicts. Ensure that R_HOME points to the correct and intended R version.

D. Review PyCharm Logs

PyCharm maintains logs that can provide insights into errors. Navigate to Help > Show Log in Explorer to access the logs.

7. Consider Using Python's Native ICA Implementations

If integration with R via rpy2 proves too cumbersome, alternative approaches within Python may offer smoother workflows.

Steps:

1. Use scikit-learn's FastICA:
   • scikit-learn provides a robust implementation of FastICA.
   • Install scikit-learn if not already installed:

   pip install scikit-learn

2. Example Usage:

   import numpy as np
   from sklearn.decomposition import FastICA
   
   # Sample data: mixed signals
   S = np.c_[heart_sound, lung_sound].T
   
   # Initialize FastICA
   ica = FastICA(n_components=2, random_state=0)
   
   # Fit and transform the data
   S_ = ica.fit_transform(S)  # Recovered signals
   
   print("Separated Signals:")
   print(S_)

Advantages:

• Seamless integration within Python environments.
• Active support and extensive documentation.

8. Final Recommendations

Integrating R functions into Python using rpy2 offers powerful capabilities but necessitates meticulous configuration, especially within integrated development environments like PyCharm on Windows systems. By following the outlined steps, the underlying issues hindering the execution of rpy2 within PyCharm should be effectively addressed.

For persistent challenges, consulting the following resources is advisable:

• rpy2 Documentation: rpy2 Official Documentation
• Community Support: Platforms such as Stack Overflow and the rpy2 GitHub Repository provide valuable assistance.
• Alternative Libraries: Exploring Python-native libraries like scikit-learn's FastICA can offer streamlined alternatives for Independent Component Analysis.

Multi-Source Data Interface: R Packages

Overview of R Packages in the Multi-Source Data Interface

Project nGene.org incorporates R packages as a critical component of its multi-source data interface, facilitating advanced statistical analysis, data visualization, and computational modeling. R, a language and environment for statistical computing and graphics, offers a vast ecosystem of packages that extend its capabilities. This integration enables the project to leverage sophisticated statistical methods and graphical tools essential for hemodynamic research and biomedical data analysis. The following outlines the functionality of R packages within the project, the challenges associated with their integration, and strategies to optimize their usage.

(A) Functionality and Features of R Packages

1. Advanced Statistical Analysis
   • Comprehensive Statistical Methods: R packages provide a wide range of statistical techniques, including regression analysis, time-series forecasting, survival analysis, and multivariate statistics.
   • Specialized Biomedical Applications: Packages such as survival, lme4, and nlme are tailored for medical data analysis, supporting survival models, linear mixed-effects models, and nonlinear modeling.
2. Data Visualization
   • High-Quality Graphics: R offers powerful visualization packages like ggplot2, enabling the creation of intricate and publication-quality plots.
   • Interactive Visualizations: Packages such as shiny and plotly facilitate interactive data exploration, enhancing the interpretability of complex datasets.
3. Computational Modeling
   • Simulation and Modeling Tools: R packages support the development of computational models, including stochastic simulations, differential equations, and agent-based models.
   • Integration with Other Languages: R can interface with C++, Python, and Fortran, allowing for performance optimization and integration of diverse computational tools.
4. Data Manipulation and Processing
   • Efficient Data Handling: Packages like dplyr, data.table, and tidyr streamline data manipulation tasks, enabling efficient handling of large datasets.
   • Bioinformatics Support: Specialized packages such as Bioconductor provide tools for genomic data analysis, sequence alignment, and biological pathway modeling.
5. Reproducible Research
   • Sweave and R Markdown: These tools facilitate the integration of R code within documentation, promoting transparency and reproducibility.
   • Version Control Integration: RStudio and related tools support integration with Git, enhancing collaborative development and version tracking.

(B) Integration within Project nGene.org

1. Embedding R Scripts in Python
   • rpy2 Library: The project utilizes rpy2, a Python interface to R, allowing seamless execution of R scripts and functions within Python codebases.
   • Cross-Language Data Exchange: rpy2 enables the transfer of data structures between R and Python, facilitating combined analyses.
2. Automation of Statistical Analyses
   • Scripted Workflows: R scripts automate complex statistical procedures, reducing manual intervention and ensuring consistency across analyses.
   • Batch Processing: The integration supports batch processing of datasets, essential for handling large-scale biomedical data.
3. Custom Package Development
   • Tailored Solutions: Development of custom R packages addresses specific research needs, incorporating proprietary algorithms and models.
   • Collaboration and Sharing: Custom packages can be shared within the research community, promoting collaboration and collective advancement.

(C) Issues and Limitations

1. Compatibility and Dependency Management
   • Version Conflicts: Differences in package versions and dependencies can lead to compatibility issues, affecting reproducibility and stability.
   • System Dependencies: Some R packages rely on external system libraries, complicating deployment across different environments.
2. Performance Considerations
   • Computational Overhead: R, being an interpreted language, may exhibit slower performance compared to compiled languages for certain tasks.
   • Memory Management: Handling large datasets can lead to significant memory consumption, potentially causing performance bottlenecks.
3. Error Handling and Debugging
   • Complex Error Messages: R's error messages can be cryptic, making debugging challenging, especially when interfaced through Python.
   • Silent Failures: Some functions may fail silently or return unexpected results without explicit errors, complicating validation.
4. Security Concerns
   • Execution of Untrusted Code: Running external R scripts poses a risk if the code is not thoroughly vetted, potentially leading to security vulnerabilities.
   • Package Integrity: Reliance on third-party packages introduces risks related to malicious code or compromised repositories.
5. Licensing and Intellectual Property
   • License Compatibility: Mixing code with different licenses can create legal complexities, especially when integrating proprietary and open-source components.
   • Attribution Requirements: Proper attribution and compliance with license terms are necessary to respect intellectual property rights.

(D) Strategies to Overcome Limitations

1. Robust Dependency Management
   • Use of Virtual Environments: Tools like packrat or renv in R can create isolated package environments, ensuring consistent dependencies.
   • Automated Installation Scripts: Implementing scripts to automate the setup of required packages and system libraries across environments.
2. Performance Optimization
   • Profiling and Benchmarking: Utilizing R's profiling tools to identify bottlenecks and optimize code for better performance.
   • Parallel Computing: Leveraging packages like parallel, foreach, and future to distribute computations across multiple cores or nodes.
   • Integrating Compiled Code: Writing performance-critical sections in C++ using Rcpp to enhance execution speed.
3. Enhanced Error Handling
   • Structured Logging: Implementing comprehensive logging mechanisms to capture errors and warnings systematically.
   • Validation Checks: Incorporating input validation and assertion checks to ensure data integrity before processing.
   • Debugging Tools: Utilizing R's debugging tools, such as browser(), traceback(), and IDE features for step-by-step execution.
4. Security Measures
   • Code Review and Auditing: Establishing protocols for reviewing R scripts and packages before integration to detect potential security issues.
   • Sandboxing Execution: Running R code within restricted environments to limit the impact of any malicious code execution.
   • Package Verification: Using tools like packrat to verify package sources and integrity before installation.
5. Licensing Compliance
   • License Auditing: Keeping detailed records of package licenses to ensure compatibility and compliance with project policies.
   • Legal Consultation: Engaging with legal experts to navigate complex licensing scenarios and intellectual property concerns.
   • Attribution Practices: Maintaining proper documentation and attribution in accordance with the licenses of used packages.
6. Documentation and Reproducibility
   • Comprehensive Documentation: Maintaining detailed documentation of all R scripts, functions, and workflows for transparency.
   • Version Control Practices: Using Git and repositories like GitHub or GitLab to track changes and collaborate effectively.
   • Reproducible Environments: Sharing environment specifications and setup instructions to facilitate reproducibility by other researchers.

(E) Conclusion

The integration of R packages within Project nGene.org significantly enhances the project's capabilities in statistical analysis, data visualization, and computational modeling. While challenges related to compatibility, performance, error handling, security, and licensing exist, strategic approaches can effectively mitigate these issues. By implementing robust dependency management, optimizing performance, enhancing error handling, enforcing security measures, ensuring licensing compliance, and promoting reproducibility, the project can fully leverage the strengths of R packages.

This integration not only supports the project's immediate research objectives but also contributes to the broader scientific community by facilitating advanced analyses and fostering collaborative innovation. The careful management of R packages within the multi-source data interface exemplifies a commitment to technical excellence, ethical standards, and scholarly rigor in advancing hemodynamic research and biomedical data analysis.

Direct Integration of R Scripts in Python: Executing FastICA.R for Signal Separation

In the pursuit of advancing and streamlining signal processing workflows, two new Python modules have been developed: nGene_rpy2 and nGene_Waveform. These modules are designed to facilitate the integration of R scripts into Python applications and to efficiently manage waveform data. The following sections provide an overview of these modules and demonstrate their practical applications.

(A) nGene_rpy2: Bridging R and Python

nGene_rpy2 is a Python class that utilizes the rpy2 library to seamlessly integrate R scripts and packages into Python applications. This class enables the loading of R code from files or strings, the importation of R packages, and the invocation of R functions directly from Python. Such integration is instrumental in leveraging R's advanced statistical and signal processing capabilities within a Python environment.

• Load R Scripts: Facilitates the loading of R code from files or strings, creating callable modules within Python.
• Import R Packages: Manages the importation of R packages, including automatic installation if packages are not already present.
• Call R Functions: Executes R functions from loaded scripts or packages with automatic data conversion between R and Python data structures.
• Execute R Commands: Allows the execution of raw R commands directly from Python, providing flexibility in scripting.

The following example demonstrates the utilization of nGene_rpy2 to perform Independent Component Analysis (ICA) using R's fastICA function:

(B) nGene_Waveform: Handling Audio Signals

nGene_Waveform is a Python class dedicated to the processing and visualization of waveform data. This class simplifies tasks such as reading audio files, normalizing signals, saving audio data, and plotting waveforms. It is essential for applications involving audio signal processing, particularly within the context of biomedical signals like heart and lung sounds.

• Read and Normalize Audio Data: Facilitates the loading of .wav files and normalization of audio signals for processing.
• Save Audio Signals: Enables the saving of processed or separated signals as .wav files.
• Plot Waveforms: Provides visualization of audio signals using matplotlib for analysis and presentation purposes.
• Handle Multiple Signals: Efficiently manages and processes multiple audio signals or components.

The example below illustrates how to utilize nGene_Waveform to read audio files and plot their waveforms:

(C) main.py: Performing ICA on Audio Signals

main.py serves as an example script demonstrating the application of the nGene_rpy2 and nGene_Waveform classes to perform Independent Component Analysis (ICA) on mixed audio signals, such as heart and lung sounds. Users may adapt this script to their specific requirements by renaming it accordingly.

• Initialize Handlers: Establishes instances of nGene_rpy2 and nGene_Waveform.
• Load and Normalize Audio Data: Reads heart and lung sound files and normalizes the signals.
• Mix Signals: Combines the original signals using a predefined mixing matrix.
• Perform ICA: Utilizes R's fastICA function to separate the mixed signals into independent components.
• Plot and Save Results: Visualizes the original, mixed, and separated signals, and saves the separated components as .wav files.

The complete script is available in main.py. Users may adapt this script as needed:

Steps Involved in Performing ICA on Audio Signals

1. Initialize Handlers:
   • Import and instantiate nGene_rpy2 and nGene_Waveform.
   • Load the R script for fastICA using nGene_rpy2.
2. Load and Normalize Audio Data:
   • Use nGene_Waveform to read and normalize heart and lung sound files.
   • Ensure both audio files have matching sample rates.
3. Mix Signals:
   • Combine the original signals using a predefined mixing matrix to create mixed signals.
4. Perform ICA:
   • Employ nGene_rpy2 to execute the fastICA function from R on the mixed signals.
   • Extract the separated independent components from the ICA result.
5. Plot and Save Results:
   • Visualize the original, mixed, and separated signals using nGene_Waveform.
   • Save the separated components as .wav files for further analysis or playback.

Adaptive Dynamic Coordinate Scaling and Visualization

Dynamic Coordinate Scaling for Mathematical Simulation

Simulating mathematical models frequently involves plotting functions or data points within a specified x-range. However, effectively visualizing these plots within a given canvas requires dynamically adjusting the y-axis to fit the function’s output range onto the screen. Techniques employed by tools like R and other mathematical software can automatically determine appropriate y-axis scaling. The methodologies described here explain how to achieve similar adaptive scaling for both the x and y axes, ensuring that the entire model is represented proportionally on the canvas.

1. Calculating Function Value Ranges

When simulating mathematical functions, the range of y-values often needs to be determined automatically after specifying the x-range to fully display the function within the visualization area. This involves:

1. Defining the x-axis domain (minimum and maximum x-values).
2. Sampling the function to find corresponding y-values.
3. Determining the minimum and maximum y-values for the given x-domain.

Once these ranges are known, a transformation can be applied that maps these data ranges to the pixel coordinates of the canvas.

Mathematical Model

Assume a function f(x) is defined over an x-range of [xMin, xMax]. The domain of the y-axis, [yMin, yMax], is determined by evaluating the function over this domain:

$$ y_{\text{values}} = \{ f(x) \mid x \in [xMin, xMax] \} $$
$$ yMin = \min(y_{\text{values}}), \quad yMax = \max(y_{\text{values}}) $$

With these ranges, a linear transformation is used to map the domain space to the canvas space:

For a canvas of width canvasWidth and height canvasHeight:

$$ \text{pixelX} = \left(\frac{x - xMin}{xMax - xMin}\right) \times \text{canvasWidth} $$
$$ \text{pixelY} = \left(1 - \frac{f(x) - yMin}{yMax - yMin}\right) \times \text{canvasHeight} $$

This transformation ensures that the function’s domain and range are mapped appropriately to the canvas coordinate system, with xMin mapping to pixel x = 0 and xMax mapping to pixel x = canvasWidth, similarly adjusting yMin and yMax along the y-axis.

// Define the function to be plotted
function f(x) {
    return Math.sin(x); // Example: a simple sine wave
}

// Define the domain of the function
const xMin = 0;
const xMax = 2 * Math.PI; // For one period of the sine wave

// Sample the function to find y-values
const samples = 1000; // Number of points to sample
let yValues = [];
for (let i = 0; i <= samples; i++) {
    const x = xMin + (i * (xMax - xMin) / samples);
    yValues.push(f(x));
}

// Calculate y-range
const yMin = Math.min(...yValues);
const yMax = Math.max(...yValues);

// Assuming a canvas context is obtained
const canvas = document.getElementById('myCanvas');
const ctx = canvas.getContext('2d');

function transformX(x) {
    return ((x - xMin) / (xMax - xMin)) * canvas.width;
}

function transformY(y) {
    return (1 - (y - yMin) / (yMax - yMin)) * canvas.height;
}

// Plot the function
ctx.beginPath();
for (let i = 0; i <= samples; i++) {
    const x = xMin + (i * (xMax - xMin) / samples);
    const y = f(x);
    if (i === 0) {
        ctx.moveTo(transformX(x), transformY(y));
    } else {
        ctx.lineTo(transformX(x), transformY(y));
    }
}
ctx.stroke();    

In this example, the f(x) function is a sine wave. The code samples points along the x-domain, calculates corresponding y-values, and then draws the function onto the canvas using the coordinate transformation functions transformX and transformY. This ensures that the entire sine wave is scaled and displayed appropriately across the available canvas space.

2. Automatic Scaling of Y-Axis for a Fixed X-Domain

When only the x-domain is specified, and there is uncertainty about the magnitude of the function’s output, automatic scaling methods can be implemented. These methods are not only for analyzing data, but also for plotting mathematical functions with unknown or dynamic ranges.

1. Data-Driven Scaling: As shown above, evaluate the function over the domain, find the function’s minimum and maximum, and scale accordingly.
2. Padding and Margins: To improve readability, padding can be added around the domain. For example, adding 10% padding ensures that the top and bottom edges of the range are not exactly at the canvas boundaries.
3. Dynamic Adjustments: For interactive applications where the x-domain might change or when the function's behavior evolves (such as parameter changes in real-time simulations), the scaling can be recalculated dynamically to fit new function values within the canvas.

Mathematical Model for Padding

Adding padding ensures that the graph does not touch the edges of the canvas for the computed y-range. Given a padding fraction p (e.g., 0.1 for 10%):

$$ \text{padding} = p \times (yMax - yMin) $$
$$ yMin_{\text{padded}} = yMin - \text{padding} \quad \text{and} \quad yMax_{\text{padded}} = yMax + \text{padding} $$

Updating the coordinate transformation function accordingly ensures that the graph is displayed with adequate margins.

// Define padding fraction
const paddingFraction = 0.1;
const padding = paddingFraction * (yMax - yMin);
const yMinPadded = yMin - padding;
const yMaxPadded = yMax + padding;

function transformY(y) {
    return (1 - (y - yMinPadded) / (yMaxPadded - yMinPadded)) * canvas.height;
}

// Redraw the function with padded y-range
ctx.clearRect(0, 0, canvas.width, canvas.height);
ctx.beginPath();
for (let i = 0; i <= samples; i++) {
    const x = xMin + (i * (xMax - xMin) / samples);
    const y = f(x);
    if (i === 0) {
        ctx.moveTo(transformX(x), transformY(y));
    } else {
        ctx.lineTo(transformX(x), transformY(y));
    }
}
ctx.stroke();    

By applying these transformations, the plotted function will appear centered and scaled on the canvas, ensuring a visually clear representation.

3. Comparing to Established Methods and Tools

Mathematical and statistical software packages (like R, Python’s matplotlib, or D3.js) provide built-in functionalities to automatically determine and adjust axis ranges. These tools typically perform:

• Automatic computation of data bounds (both x and y ranges).
• Adaptive adjustment of plot view based on the computed ranges and available display area.
• Application of aesthetic considerations like padding, aspect ratios, and margin adjustments for clarity.

In JavaScript, similar functionalities can be replicated by evaluating the function or dataset to be plotted, determining the necessary scaling, and then rendering the result on an HTML canvas or SVG using appropriate transformations as demonstrated.

4. Dynamic Updates for Real-Time Simulations

For real-time simulations or interactive applications where parameters of the mathematical model may change dynamically, the plot’s x and y axes need to be recalculated and redrawn regularly. This can be achieved by:

1. Recalculating the function values whenever input parameters change.
2. Updating yMin, yMax, and applying necessary transformations.
3. Redrawing the function with the updated scaling to fit the new range onto the canvas.

This ensures that the visualization remains accurate and readable throughout simulation changes.

Dynamic Range Adjustment Techniques for Graph Visualization in JavaScript

Designing a JavaScript simulator that adaptively adjusts the x and y axes ranges can significantly improve the clarity and user experience of dynamic data visualizations. This document outlines several methodologies for setting these ranges flexibly, including mathematical models, algorithmic approaches, and example JavaScript implementations. Implementing these methods will enable the simulator to adjust automatically to data changes and user interactions while maintaining a formal, clear, and refined style.

1. Automatic Scaling Based on Data Extremes

Automatic scaling adjusts axis ranges according to the minimum and maximum values of the dataset. This involves calculating these extremes, optionally adding padding for clarity, and re-rendering the graph.

Mathematical Model:

If dataMin and dataMax represent the minimum and maximum data values along an axis, the axis range can be calculated as:

$$ \text{axisMin} = \text{dataMin} - \text{padding} \quad \text{and} \quad \text{axisMax} = \text{dataMax} + \text{padding} $$

Where padding can be a fixed value or a fraction of the range. For example, a 10% padding on the data range along the y-axis can be computed as:

$$ \text{padding} = 0.1 \times (\text{dataMax} - \text{dataMin}) $$

function autoScaleAxis(data) {
    const dataMin = Math.min(...data);
    const dataMax = Math.max(...data);
    const padding = 0.1 * (dataMax - dataMin);
    const axisMin = dataMin - padding;
    const axisMax = dataMax + padding;
    return { axisMin, axisMax };
}

// Example usage:
const yValues = [12, 15, 20, 22, 29, 35];
const { axisMin, axisMax } = autoScaleAxis(yValues);
// axisMin and axisMax can now be used to set the graph's y-axis range.    

This approach ensures the graph always displays all data points within the visible range, adjusting automatically as new data arrives or as data changes.

2. Data-Driven Adaptive Scaling with Statistical Techniques

Data-driven adaptive scaling uses statistical techniques to adjust ranges dynamically, focusing on typical data values while handling outliers effectively. For example, scaling based on specific percentiles ensures that the majority of data is visible without extreme outliers dominating the view.

Mathematical Model:

To exclude outliers, the axis range might be set between the 5th and 95th percentiles of the dataset. This can be computed by sorting the data and selecting values at these percentile ranks. If sortedData is the sorted dataset:

$$ \text{lowerBound} = \text{sortedData}[0.05 \times \text{dataLength}] $$
$$ \text{upperBound} = \text{sortedData}[0.95 \times \text{dataLength}] $$

function percentileScaleAxis(data, lowerPercentile, upperPercentile) {
    const sortedData = [...data].sort((a, b) => a - b);
    const lowerIndex = Math.floor((lowerPercentile / 100) * sortedData.length);
    const upperIndex = Math.ceil((upperPercentile / 100) * sortedData.length) - 1;
    const axisMin = sortedData[lowerIndex];
    const axisMax = sortedData[upperIndex];
    return { axisMin, axisMax };
}

// Example usage:
const yValues = [12, 15, 20, 22, 29, 35, 100, -10];
const { axisMin, axisMax } = percentileScaleAxis(yValues, 5, 95);
// This sets the y-axis range between the 5th and 95th percentiles, excluding extreme outliers.    

This method ensures that graphs remain readable even when datasets contain extreme values that would otherwise dominate the visual scale.

3. User-Interactive Zooming and Panning

Interactive zoom and pan functionality allows manual adjustment of the viewing range, providing greater flexibility and control. Libraries such as D3.js or Chart.js can be employed to implement this functionality. Programmatically, the axis range can be updated based on user inputs or interactions (e.g., mouse wheel events for zooming, click-and-drag for panning).

Mathematical Model:

Assuming an initial axis range [axisMin, axisMax], zooming can be represented as scaling this range by a zoom factor z. For zooming in:

$$ \text{newRange} = (\text{oldRange}) \times \frac{1}{z} $$

For panning horizontally or vertically by a factor p of the data range:

$$ \text{newAxisMin} = \text{oldAxisMin} + p \times (\text{axisMax} - \text{axisMin}) $$
$$ \text{newAxisMax} = \text{oldAxisMax} + p \times (\text{axisMax} - \text{axisMin}) $$

// Using D3.js for zoom and pan
const svg = d3.select('svg');
const zoom = d3.zoom()
    .scaleExtent([1, 10]) // Allows 1x to 10x zoom
    .translateExtent([[-100, -100], [width + 100, height + 100]]) // Limit panning
    .on('zoom', zoomed);

function zoomed(event) {
    // Event.transform contains scale and translation
    svg.attr('transform', event.transform);
}

// Applying zoom behavior to SVG
svg.call(zoom);    

With zoom and pan, users can focus on specific data segments or explore the graph in more detail without modifying the underlying data or redrawing the entire graph.

4. Dynamic Range Adjustments with Smoothing Intervals

Smoothing functions (e.g., moving averages or exponential smoothing) can be used to determine the axis range based on a smoothed representation of the data. This results in a more stable range over time, avoiding abrupt changes in axis scales when dealing with real-time data.

Mathematical Model:

A simple moving average for a window of size w can be applied to the data values. If data[i] represents the ith data point:

$$ \text{smoothedData}[i] = \frac{1}{w} \sum_{j=i-w+1}^{i} \text{data}[j] $$

The axis range is then computed from this smoothed data:

$$ \text{axisMin} = \min(\text{smoothedData}) \quad \text{and} \quad \text{axisMax} = \max(\text{smoothedData}) $$

function movingAverage(data, windowSize) {
    if (windowSize <= 1) return data;
    let result = [];
    for (let i = 0; i < data.length; i++) {
        let start = Math.max(0, i - windowSize + 1);
        const windowData = data.slice(start, i + 1);
        const average = windowData.reduce((sum, val) => sum + val, 0) / windowData.length;
        result.push(average);
    }
    return result;
}

// Example usage:
const yValues = [12, 15, 20, 22, 29, 35, 100, 90, 85, 80];
const smoothedYValues = movingAverage(yValues, 3);
const axisMin = Math.min(...smoothedYValues);
const axisMax = Math.max(...smoothedYValues);    

Dynamic range adjustments based on smoothed values produce more stable and visually consistent graphs, especially useful in scenarios where data points can fluctuate rapidly.

5. Real-Time Range Adjustments for Streaming Data

For data that updates continuously, the range can be adjusted in real-time based on the most recent subset of data. This involves continuously recalculating the minimum and maximum values over a rolling window of data points.

Mathematical Model:

If data is streamed in sequences and only the last n data points are considered:

$$ \text{rollingData} = \text{dataPoints}[\text{dataLength} - n, \dots, \text{dataLength} - 1] $$
$$ \text{axisMin} = \min(\text{rollingData}) \quad \text{and} \quad \text{axisMax} = \max(\text{rollingData}) $$

function realTimeScaleAxis(data, windowSize) {
    const start = Math.max(0, data.length - windowSize);
    const recentData = data.slice(start);
    const dataMin = Math.min(...recentData);
    const dataMax = Math.max(...recentData);
    return { axisMin: dataMin, axisMax: dataMax };
}

// Example usage:
let yValues = [12, 15, 20, 22, 29, 35, 28, 30, 50]; // streaming data
const windowSize = 5;
const { axisMin, axisMax } = realTimeScaleAxis(yValues, windowSize);
// axisMin and axisMax are calculated based on the last 5 data points.    

Real-time scaling ensures that graphs reflect the latest data trends while maintaining a focus on the most relevant and recent information.

6. Responsive Range Adjustment Based on Display Dimensions

Responsive scaling adjusts the range and resolution of the graph based on its display dimensions. This ensures the graph maintains clarity and proportion across various screen sizes or container dimensions.

Mathematical Model:

This approach involves mapping data ranges to pixel dimensions adaptively:

$$ \text{pixelsPerUnit} = \frac{\text{graphWidth (or height)}}{\text{axisMax} - \text{axisMin}} $$

function responsiveAxisRange(data, containerWidth, containerHeight) {
    const dataMin = Math.min(...data);
    const dataMax = Math.max(...data);
    // For simplicity, assign axes based on container aspect ratio
    const aspectRatio = containerWidth / containerHeight;
    const range = dataMax - dataMin;
    const axisMin = dataMin - 0.1 * range;
    const axisMax = dataMax + 0.1 * range;
    return { axisMin, axisMax, aspectRatio };
}

// Example usage:
const containerWidth = 800;
const containerHeight = 600;
const yValues = [12, 15, 20, 22, 29, 35];
const { axisMin, axisMax, aspectRatio } = responsiveAxisRange(yValues, containerWidth, containerHeight);
// The graph can now be drawn proportionally within these dimensions.    

Responsive range adjustments maintain visual clarity and aesthetic consistency, regardless of the platform or device dimensions.

7. Algorithmic Scaling Techniques for Advanced Control

Advanced algorithmic scaling techniques use additional computations to determine the optimal range, such as outlier detection, data clustering, or binning. These methods ensure that the graph represents the most relevant data patterns while managing outliers or large datasets efficiently.

Mathematical Model:

An example of outlier detection using the Z-score method:

$$ Z = \frac{x - \mu}{\sigma} $$

Where μ is the mean and σ is the standard deviation of the data. Data points with |Z| greater than a chosen threshold (commonly 3) are considered outliers and are excluded from the range calculation.

function removeOutliers(data, threshold = 3) {
    const mean = data.reduce((a, b) => a + b, 0) / data.length;
    const std = Math.sqrt(data.reduce((a, b) => a + Math.pow(b - mean, 2), 0) / data.length);
    return data.filter(value => Math.abs((value - mean) / std) < threshold);
}

// Example usage:
const yValues = [12, 15, 20, 22, 29, 35, 100, -50];
const filteredData = removeOutliers(yValues, 3);
const axisMin = Math.min(...filteredData);
const axisMax = Math.max(...filteredData);    

Algorithmic scaling techniques maintain the readability and accuracy of the graph by focusing on data distributions and effectively handling outliers.

8. AI or Machine Learning-Based Range Prediction

AI or machine learning algorithms can be used to predict future data trends and adjust axis ranges proactively. This approach is beneficial for graphs visualizing highly fluctuating data, where adaptive predictive adjustments can enhance the user experience.

Mathematical Model:

Using a linear regression model:

$$ \hat{y} = \beta_0 + \beta_1 x $$

Where β₀ and β₁ are coefficients determined by a training process using available data. Predictions ŷ can then set the future axis range adaptively.

function linearRegression(dataPoints) {
    const xValues = dataPoints.map((d, i) => i);
    const yValues = dataPoints;
    const xMean = xValues.reduce((a, b) => a + b) / xValues.length;
    const yMean = yValues.reduce((a, b) => a + b) / yValues.length;

    let numerator = 0, denominator = 0;
    for (let i = 0; i < xValues.length; i++) {
        numerator += (xValues[i] - xMean) * (yValues[i] - yMean);
        denominator += Math.pow(xValues[i] - xMean, 2);
    }
    const beta1 = numerator / denominator;
    const beta0 = yMean - beta1 * xMean;

    return { beta0, beta1 };
}

// Example usage:
const yValues = [12, 15, 20, 22, 29, 35];
const { beta0, beta1 } = linearRegression(yValues);
const lastIndex = yValues.length - 1;
const predictedextValue = beta0 + beta1 * (lastIndex + 1);    

Machine learning-based range adjustments can be especially powerful for complex data patterns, allowing the graph to preemptively adjust for anticipated fluctuations.

Summary

Each method outlined above provides a unique strategy for adjusting the x and y axes ranges dynamically in a JavaScript graph or simulator:

1. Automatic Scaling Based on Data Extremes: Adapts ranges to the minimum and maximum data values.
2. Data-Driven Adaptive Scaling with Statistical Techniques: Utilizes percentiles or other statistical measures to handle outliers.
3. User-Interactive Zooming and Panning: Allows manual control over graph magnification and viewed data segments.
4. Dynamic Range Adjustments with Smoothing Intervals: Uses smoothed data values to create stable and visually coherent ranges.
5. Real-Time Range Adjustments for Streaming Data: Continuously recalculates the range based on the most recent subset of data.
6. Responsive Range Adjustment Based on Display Dimensions: Ensures graph clarity and proportionality across different screen sizes.
7. Algorithmic Scaling Techniques for Advanced Control: Employs outlier detection and other algorithms to manage extreme or large datasets.
8. AI or Machine Learning-Based Range Prediction: Forecasts data trends and adjusts ranges proactively based on predictive models.

Implementing one or combining several of these methodologies will result in adaptive and user-friendly graphing solutions. Each approach offers distinct advantages depending on the context, data nature, and user requirements.

Using nGeneDynamicCoordinate: structure, usage, and extensibility (Written November 23, 2025)

I. Conceptual role of nGeneDynamicCoordinate

The nGeneDynamicCoordinate class plays the role of a reusable, model-agnostic “view adapter” between a hemodynamic model and the SVG plotting space. The class does not know the physiological meaning of the waveforms; instead, it receives a model that can supply values over time (for example, pressure and velocity) and converts these values into pixel coordinates, time ticks, and “nice” axis scales.

In the LBSM v0.2.01 implementation, nGeneDynamicCoordinate handles the following responsibilities:

• Determining the horizontal time domain (tMin, tMax) for plotting, based on the model.
• Sampling the model’s waveforms to find positive and negative ranges for pressure and velocity.
• Splitting the vertical space into a negative band (preview) and a positive band (main waveform) around a baseline.
• Providing mapping functions from time/value to SVG coordinates.
• Generating “nice” tick values for grid and axis labels.

Because all of this logic is encapsulated in a single class, the same dynamic coordinate system can be reused across multiple models, as long as the new models respect a simple interface (time domain and waveform evaluators).

II. Structure and interface of the class

Constructor and core fields

1. The constructor accepts three arguments: the plotting rectangle, a model instance, and optional settings. The essential part of the implementation is as follows:

   class nGeneDynamicCoordinate{
     constructor(plotRect, model, options = {}){
       this.plot = plotRect;
       this.model = model;
       this.negBandFraction = options.negBandFraction ?? NEG_BAND_FRACTION;
       this.dtSample = options.dtSample ?? 0.001;
       this.padFraction = options.padFraction ?? 0.06;
       this.recompute();
     }
     ...
   }

   In this design:

   • plotRect is an object such as { x0:55, y0:30, w:640, h:540 }, describing the SVG region used for the chart.
   • model is any object that provides at least p(t), v(t), tMin(), and tMax() (in LBSM, this is an nGeneLBSM instance).
   • negBandFraction sets how much vertical space is reserved for the “negative preview band” above the baseline.
   • dtSample is the sampling step in seconds used to scan the model for its extrema.
   • padFraction adds a small margin so that waveforms do not touch the frame.

   The constructor finishes by calling this.recompute(), which synchronizes all internal scaling parameters to the current model state.

Sampling the model and computing scales

1. The core of the dynamic behavior resides in the recompute() method. This method queries the model for its time domain, samples the waveforms, and derives the scale factors:

   recompute(){
     this.xMin = this.model.tMin();
     this.xMax = this.model.tMax();
   
     if (!(Number.isFinite(this.xMin) && Number.isFinite(this.xMax)) || this.xMax <= this.xMin){
       this.xMin = 0;
       this.xMax = 1;
     }
   
     this.xScale = this.plot.w / (this.xMax - this.xMin);
   
     this.hNeg = this.plot.h * this.negBandFraction;
     this.hPos = this.plot.h - this.hNeg;
     this.yBase = this.plot.y0 + this.hNeg;
   
     let pMaxPos = 0, vMaxPos = 0, pMaxNeg = 0, vMaxNeg = 0;
     const dt = this.dtSample;
   
     for (let t = this.xMin; t <= this.xMax + 1e-12; t += dt){
       const p = this.model.p(t);
       const v = this.model.v(t);
       if (p < 0) pMaxNeg = Math.max(pMaxNeg, -p); else pMaxPos = Math.max(pMaxPos, p);
       if (v < 0) vMaxNeg = Math.max(vMaxNeg, -v); else vMaxPos = Math.max(vMaxPos, v);
     }
   
     const pad = this.padFraction;
     this.pMaxPos = (pMaxPos || 1) * (1 + pad);
     this.vMaxPos = (vMaxPos || 1) * (1 + pad);
     this.pMaxNeg = pMaxNeg * (1 + pad);
     this.vMaxNeg = vMaxNeg * (1 + pad);
   
     this.pScalePos = this.hPos / Math.max(1e-9, this.pMaxPos);
     this.vScalePos = this.hPos / Math.max(1e-9, this.vMaxPos);
     this.pScaleNeg = this.hNeg / Math.max(1e-9, this.pMaxNeg || 1);
     this.vScaleNeg = this.hNeg / Math.max(1e-9, this.vMaxNeg || 1);
   
     this.tTicks = [];
     const step = 0.05;
     for (let t = Math.ceil(this.xMin/step)*step; t <= this.xMax + 1e-12; t += step){
       this.tTicks.push(+t.toFixed(2));
     }
   }

   The essential steps for any future model are:

   • Use model.tMin() and model.tMax() as the horizontal domain.
   • Sample model.p(t) and model.v(t) over that domain to find positive and negative ranges.
   • Split the vertical space into a negative band (hNeg) and a positive band (hPos) around yBase.
   • Convert value ranges into pixel-per-unit scale factors for pressure and velocity, positive and negative.
   • Populate tTicks with human-friendly time ticks for rendering the grid.

Mapping utilities and “nice” scales

1. Once the sampling and internal scaling are ready, nGeneDynamicCoordinate exposes three primary mapping functions and a set of helpers for axis ticks:

   tToX(t){
     return this.plot.x0 + (t - this.xMin) * this.xScale;
   }
   
   pToY(p){
     return (p >= 0)
       ? (this.yBase + p * this.pScalePos)
       : (this.yBase - (-p) * this.pScaleNeg);
   }
   
   vToY(v){
     return (v >= 0)
       ? (this.yBase + v * this.vScalePos)
       : (this.yBase - (-v) * this.vScaleNeg);
   }

   For axes and gridlines, the class offers ready-to-use tick sets:

   getTimeTicks(){
     return this.tTicks.slice();
   }
   
   getPressureNiceScales(){
     const pPosNS = nGeneDynamicCoordinate.niceScale(0, this.pMaxPos, 8);
     const pNegNS = nGeneDynamicCoordinate.niceScale(0, this.pMaxNeg, 6);
     return { pPosNS, pNegNS };
   }
   
   getVelocityNiceScales(){
     const vPosNS = nGeneDynamicCoordinate.niceScale(0, this.vMaxPos, 8);
     const vNegNS = nGeneDynamicCoordinate.niceScale(0, this.vMaxNeg, 6);
     return { vPosNS, vNegNS };
   }

   The static niceScale helper operates on a numeric range and returns a struct containing min, max, step, and ticks, which are then used for grid drawing and axis labels.

III. Usage in LBSM v0.2.01

In LBSM v0.2.01, nGeneDynamicCoordinate is used in combination with nGeneLBSM to render the pressure and velocity waveforms interactively. The pattern serves as a canonical example of how to connect a model to this coordinate system.

Instantiating model and coordinate system

1. The key lines appear in the init() function:

   model = new nGeneLBSM({
     Alpha:  parseFloat(sliders.Alpha.value),
     L:      parseFloat(sliders.L.value),
     GammaR: parseFloat(sliders.GammaR.value),
     ET:     parseFloat(sliders.ET.value),
     R:      parseFloat(sliders.R.value),
     GammaL: parseFloat(sliders.GammaL.value),
     Ap:     parseFloat(sliders.Ap.value),
     C:      Number(sliders.C.value),
     Gamma:  parseFloat(sliders.Gamma.value),
     As:     parseFloat(sliders.As.value),
     Beta:   parseFloat(sliders.Beta.value)
   });
   
   coord = new nGeneDynamicCoordinate(PLOT, model, {
     negBandFraction: NEG_BAND_FRACTION
   });

   Here, PLOT is the static SVG geometry, while the model instance provides the physiological waveforms. From this point onward, every part of the rendering code interacts with the SVG only through coord and model, rather than directly manipulating raw scales.

Drawing axes and grid with nGeneDynamicCoordinate

1. The drawAxes() function uses coord to position the baseline, time gridlines, and pressure/velocity ticks:

   // baseline (0) position and extent
   baseline.setAttribute('x1', PLOT.x0);
   baseline.setAttribute('x2', PLOT.x0 + PLOT.w);
   baseline.setAttribute('y1', coord.yBase);
   baseline.setAttribute('y2', coord.yBase);
   
   // time grid/labels (top)
   const tTicks = coord.getTimeTicks();
   for (const t of tTicks){
     const x = coord.tToX(t);
     ...
   }

   Pressure ticks and velocity ticks similarly call coord.getPressureNiceScales() and coord.getVelocityNiceScales(), then map values to y-coordinates via coord.pToY() and coord.vToY(). This ensures that all axes automatically adapt whenever the model’s domain or amplitude changes.

Building SVG paths for waveforms

1. For actual waveforms, the helper buildPathSigned uses coord.tToX() and coord.vToY() / coord.pToY() to translate model values into SVG path commands:

   ghostVelPre.setAttribute(
     'd',
     buildPathSigned(
       coord.xMin,
       0,
       t => model.v(t),
       v => coord.vToY(v),
       false
     )
   );
   
   pathPre.setAttribute(
     'd',
     buildPathSigned(
       0,
       coord.xMax,
       t => model.p(t),
       p => coord.pToY(p),
       false
     )
   );

   Because the path builder takes callbacks rather than directly referencing the model or scale, this pattern remains usable for any other model that follows the same p(t)/v(t) convention. Only the model instance and the coordinate object need to be changed.

Crosshair and marker placement

1. The interactive crosshair and the PAcT/TTPSP markers also rely on coord to maintain consistency:

   function setCrosshairAtT(t){
     const x  = coord.tToX(t);
     const yv = coord.vToY(model.v(t));
     const yp = coord.pToY(model.p(t));
     setCrosshair(x, yv, yp);
     ro.t.textContent = `t: ${t.toFixed(3)} s`;
     ro.v.textContent = `velocity: ${model.v(t).toFixed(3)} m/s`;
     ro.p.textContent = `pressure: ${model.p(t).toFixed(1)} mmHg`;
   }

   The same approach is used in drawMarkers() for PAcT and TTPSP dots, by first asking the model for marker times (model.PAcT(), model.TTPSP()) and then mapping to coordinates with coord.tToX(), coord.vToY(), and coord.pToY().

IV. Applying nGeneDynamicCoordinate to other models

The design of nGeneDynamicCoordinate is sufficiently general to be reused in other contexts beyond the logistic-based systolic model. The main requirement is that the new model provides time-domain information and two scalar waveforms that can be conceptually mapped to “pressure-like” and “velocity-like” channels. The physiological meaning may differ; the coordinate system remains agnostic.

Model contract required by nGeneDynamicCoordinate

1. In its current form, nGeneDynamicCoordinate expects a model with the following interface:

   Method	Purpose
   tMin()	Return the starting time (in seconds) for the domain to be plotted.
   tMax()	Return the ending time for the domain.
   p(t)	Return the “pressure-like” quantity at time t.
   v(t)	Return the “velocity-like” quantity at time t.

   In LBSM, these methods are provided by nGeneLBSM. For a different application (for example, systemic arterial waveforms or synthetic test signals), the new model can implement these methods with different internal formulas.

Example: sinusoidal test model with nGeneDynamicCoordinate

1. The following example shows how a simple sinusoidal model can be plugged into nGeneDynamicCoordinate for testing or for visualizing non-logistic dynamics. The “pressure” and “velocity” labels are reused as generic channels:

   class nGeneSinusModel{
     constructor(T = 1.0){
       this.T = T;           // total duration
       this.A_p = 30;        // amplitude for "pressure"
       this.A_v = 2;         // amplitude for "velocity"
     }
   
     tMin(){ return 0.0; }
     tMax(){ return this.T; }
   
     // Simple sine-based "pressure"
     p(t){
       const omega = 2 * Math.PI / this.T;
       return this.A_p * Math.sin(omega * t);
     }
   
     // Simple cosine-based "velocity"
     v(t){
       const omega = 2 * Math.PI / this.T;
       return this.A_v * Math.cos(omega * t);
     }
   }

   This model can then be wired to the coordinate system in a manner analogous to LBSM:

   const PLOT = { x0:55, y0:30, w:640, h:540 };
   
   const model = new nGeneSinusModel(1.0);
   const coord = new nGeneDynamicCoordinate(PLOT, model, {
     negBandFraction: 0.15,
     dtSample: 0.001,
     padFraction: 0.10
   });
   
   // Example of drawing a path for "pressure"
   const pathD = buildPathSigned(
     coord.xMin,
     coord.xMax,
     t => model.p(t),
     p => coord.pToY(p),
     false
   );

   The same drawAxes(), buildPathSigned(), and crosshair logic from LBSM can be reused; only the model instantiation changes. This demonstrates how nGeneDynamicCoordinate can serve as a foundation for a unified plotting framework across multiple signal families.

Adapting to models with different semantics

1. For models that do not naturally produce “pressure” and “velocity” but instead have arbitrary channels (for example, “flow” and “area” or “signal A” and “signal B”), several adaptation strategies are possible:

   • Map the primary channel to p(t) and the secondary channel to v(t), keeping the current class unchanged.
   • Extend the class to accept an array of channels and rename methods to generic y1(t), y2(t), etc., while retaining internal logic.
   • Subclass nGeneDynamicCoordinate and override sampling logic so that the negative band and positive band correspond to different subsets of signals.

   Because nGeneDynamicCoordinate is already decoupled from the specific formulas of the LBSM, the changes needed for such adaptation are limited and largely local to the coordinate class.

V. Practical adaptation tips

When to recompute coordinates

1. Whenever the underlying model parameters change (for example, sliders for ET, resistance, or compliance), the coordinate system must be updated. In LBSM, this is handled inside the slider event listener:

   sl.addEventListener('input', ()=>{
     const val = (k === 'C') ? Number(sl.value) : parseFloat(sl.value);
     model.setParam(k, val);
     out.textContent = fmtOut(k, model.getParam(k));
     coord.recompute();
     renderStatic(); // immediate redraw with updated scales
   });

   The important pattern for other models is to call coord.recompute() after any structural change in the waveform, followed by a redraw of axes and paths.

Performance considerations

1. The sampling resolution dtSample controls the trade-off between accuracy and performance. For high-frequency or rapidly varying waveforms, a smaller dtSample may be required, while for smoother waveforms, a coarser step can significantly reduce computation cost:

   • For echo-like systolic waveforms, dtSample = 0.001 (1 ms) often provides smooth curves.
   • For synthetic or low-variability signals, dtSample = 0.002–0.005 may be adequate.
   • For long-duration simulations (many seconds), it may be beneficial to decimate the sampling or dynamically adjust the step size.

   These adjustments can be made without changing any of the surrounding rendering code, simply by passing a different dtSample value to the nGeneDynamicCoordinate constructor.

Using the negative band effectively

1. The current implementation reserves a portion of the vertical space as a “negative band” for previewing negative lobes and pre-ejection behavior. For other models, this region could be repurposed in various ways:

   • Previewing pre-stimulus baselines or pre-trigger oscillations.
   • Displaying inverted or residual signals for diagnostic purposes.
   • Highlighting error or difference signals (for example, model–data residuals) while preserving the main waveforms in the positive band.

   By tuning negBandFraction, the vertical layout can be adapted to the importance of the negative portion of the signal in a given application.

Generalization roadmap

1. For a broader nGene ecosystem, a gradual generalization path may be considered:

   • Step 1: Continue using nGeneDynamicCoordinate as-is for all two-channel time series that can be interpreted as “pressure-like” and “velocity-like”.
   • Step 2: Introduce a thin compatibility layer that maps arbitrary model outputs (y1(t), y2(t)) onto p(t) and v(t) without changing the coordinate class.
   • Step 3: When needed, refactor the class internals to allow an arbitrary number of channels, while keeping the public mapping interface stable.

   This approach allows immediate reuse of the current implementation while preserving future flexibility for more complex hemodynamic or multi-signal visualization modules.

Written on November 23, 2025

How CORS failures were mitigated in nGene Web Crawler 0.1.1 (Written November 14, 2025)

The 0.1.1 release introduces a fault‑tolerant fetching pipeline designed to overcome browser‑side CORS restrictions without requiring external setup. The approach adds an Auto fetch mode that attempts a sequence of strategies—Direct → Custom Proxy (if configured) → AllOrigins → Jina Reader—and proceeds with the first workable response. In addition, HTML detection was made resilient so that proxied responses with non‑HTML Content-Type values are still parsed when they contain HTML‑like markup.

I. Summary of the solution

• Automatic fallback chain: attempts Direct fetch first; upon failure, falls back to a user‑defined proxy (if any), then to AllOrigins, and finally to Jina Reader.
• Heuristic HTML detection: parses a response as HTML when it looks like HTML, even if a proxy returns text/plain.
• HEAD→GET safety net: image metadata retrieval attempts HEAD first and falls back to a lightweight GET for servers or proxies that do not support HEAD.
• Original‑URL resolution: links and images are resolved against the original page URL so that extracted tables remain consistent and Excel‑friendly.

II. Fetch pipeline at a glance

III. Responsible code excerpts from 0.1.1

The following excerpts implement the CORS‑tolerant pipeline and related robustness improvements.

1) Auto fetcher with ordered fallbacks

function makeAutoFetcher(mode, proxyPrefix = '') {
  // Builders return a URL to request for a given target, or null if not applicable
  const builders = [];

  const directBuilder = target => target;

  const userProxyBuilder = (target) => {
    if (!proxyPrefix) return null;
    if (proxyPrefix.includes('{url}')) return proxyPrefix.replace('{url}', encodeURIComponent(target));
    if (proxyPrefix.includes('allorigins')) return proxyPrefix + encodeURIComponent(target);
    if (proxyPrefix.endsWith('http://') || proxyPrefix.endsWith('https://')) {
      const u = new URL(target);
      return proxyPrefix + u.host + u.pathname + u.search;
    }
    return proxyPrefix + encodeURIComponent(target);
  };

  const allOriginsBuilder = target => `https://api.allorigins.win/raw?url=${encodeURIComponent(target)}`;

  const jinaBuilder = target => {
    const u = new URL(target);
    const base = u.protocol === 'https:' ? 'https://r.jina.ai/https://' : 'https://r.jina.ai/http://';
    return `${base}${u.host}${u.pathname}${u.search}`;
  };

  if (mode === 'direct') {
    builders.push(directBuilder);
  } else if (mode === 'proxy') {
    if (proxyPrefix) builders.push(userProxyBuilder);
    builders.push(allOriginsBuilder, jinaBuilder); // fallbacks
  } else { // auto
    builders.push(directBuilder);
    if (proxyPrefix) builders.push(userProxyBuilder);
    builders.push(allOriginsBuilder, jinaBuilder);
  }

  async function tryFetch(buildUrl, reqInit, want) {
    const url = buildUrl(reqInit._target);
    if (!url) throw new Error('skip');
    const init = { ...reqInit };
    delete init._target;
    const res = await fetch(url, init);
    if (!res.ok && want === 'text') {
      // Allow parsing even if non-200, still helpful for links
    }
    return res;
  }

  async function withFallback(reqInit, want = 'text') {
    const errors = [];
    for (const b of builders) {
      try {
        const res = await tryFetch(b, reqInit, want);
        if (want === 'text') {
          const text = await res.text();
          return { ok: res.ok, status: res.status, headers: res.headers, text };
        } else if (want === 'blob') {
          const blob = await res.blob();
          return { ok: res.ok, status: res.status, headers: res.headers, blob };
        } else if (want === 'head') {
          return { ok: res.ok, status: res.status, headers: res.headers };
        }
      } catch (e) {
        errors.push(e.message || 'error');
        continue;
      }
    }
    throw new Error('All fetch attempts failed');
  }

  return {
    async getText(url, signal) {
      return withFallback({ _target: url, signal, redirect: 'follow' }, 'text');
    },
    async head(url, signal) {
      // Try HEAD; if all fail, fall back to GET(headers-only read)
      try {
        return await withFallback({ _target: url, method: 'HEAD', signal, redirect: 'follow' }, 'head');
      } catch {
        try {
          const r = await withFallback({ _target: url, method: 'GET', signal, redirect: 'follow' }, 'head');
          return r;
        } catch {
          return { ok: false, status: 0, headers: new Headers() };
        }
      }
    },
    async getBlob(url, signal) {
      return withFallback({ _target: url, signal, redirect: 'follow' }, 'blob');
    }
  };
}

2) Heuristic HTML detection for proxied responses

// Treat as HTML if type says html OR if the body "looks like" HTML
_looksHtml(ctype, text) {
  const ctOk = (ctype || '').includes('text/html');
  if (ctOk) return true;
  const t = (text || '').slice(0, 1024).toLowerCase();
  return /<html|<head|<body|<a\s+href=|<img\s+/i.test(t);
}

3) Crawler usage of the auto fetcher

// Initialization
this.fetcher = makeAutoFetcher(options.fetchMode, options.proxyPrefix);

// Per-page fetch with fallback
const res = await this.fetcher.getText(url, this.abort.signal);
const status = res.status || 0;
const ctype  = res.headers.get('content-type') || '';
const text   = res.text || '';

if (this._looksHtml(ctype, text)) {
  const doc = new DOMParser().parseFromString(text, 'text/html');
  // ... extract links and images ...
}

IV. Practical configuration notes

• Default mode: the interface selects Auto by default so that CORS‑safe fallbacks activate automatically when a direct request fails.
• Custom proxy: a dedicated input accepts a prefix (including {url} placeholder) for private proxies; the chain still proceeds to public fallbacks if the custom proxy is unreachable or non‑permissive.
• Consistency of extraction: all links and image URLs are resolved against the original page URL, ensuring stable rows for Excel copy‑and‑paste even when fetched through a proxy endpoint.

V. Additional robustness considerations

• robots.txt best effort: User-agent: * rules are parsed when requested, with a conservative allow/deny check based on prefix matching.
• Image metadata: an initial HEAD request is attempted for size and type; when unavailable, a fallback GET path is used, and very small files can be skipped based on a configurable byte threshold.
• Natural size probing: image naturalWidth/naturalHeight values are captured without canvas, avoiding cross‑origin pixel reads while enriching the image table.

Written on November 14, 2025

Integrating a Proxy Service into an Nginx Workflow (Written November 15, 2025)

Modern web projects often consist of multiple components such as a main application and auxiliary services. For example, Project nGene.org operates a primary website (www.ngene.org) alongside a dedicated proxy service (proxy.ngene.org). To combine these seamlessly, Nginx is employed as a reverse proxy in front of Dockerized application components. This configuration provides a robust way to manage traffic, security, and domain structure for the various services involved. The following sections discuss why a reverse proxy is beneficial, explain the concept of a CORS proxy, and compare two strategies for deploying the proxy service: using a separate subdomain versus a path under the main site.

1. Role of Reverse Proxy and Containerization

   Deploying Nginx as a reverse proxy offers several advantages in a multi-service environment. First, it avoids port conflicts by allowing all external requests to use standard HTTP/HTTPS ports. Even if the internal services (such as the main site and proxy service) run on different ports or containers, external users only interact with port 80 or 443 on the Nginx host. This forwarding through a reverse proxy removes the need to expose unconventional ports for each service.

   Second, Nginx centralizes TLS termination. The reverse proxy handles encryption (SSL/TLS) at the front, so internal traffic between Nginx and the backend services can remain within the private network using plain HTTP. This setup simplifies certificate management, since only Nginx requires an HTTPS certificate (for *.ngene.org), and all services behind it inherit secure access without individual certificates.

   Third, using a reverse proxy enables a unified domain and URL structure for end users. Visitors can access diverse functionalities through a single domain (for instance, www.ngene.org and its subpaths or subdomains) instead of dealing with separate domains or port numbers for each component. This consistency improves user experience and makes the architecture appear as one cohesive platform.

   Finally, combining Nginx with containerization (e.g., Docker) ensures each service is isolated yet easily integrated. Docker provides environment reproducibility and clean isolation for the main application and proxy service. Nginx then routes requests to the appropriate container, bridging them under a common interface. This synergy streamlines deployment by decoupling internal service configurations from the public-facing endpoints.

2. Understanding CORS and the Value of a CORS Proxy

   Web browsers enforce a security policy called Cross-Origin Resource Sharing (CORS), which restricts web pages from requesting data from a different origin (domain or port) unless explicitly allowed. In the context of Project nGene.org, this became significant because a client-side web crawler in the main application needs to fetch data from external websites. Directly fetching external URLs from JavaScript running on www.ngene.org would normally be blocked by the browser’s CORS policy if those target sites do not permit cross-origin access.

   A CORS proxy is a server-side intermediary designed to overcome this limitation. Instead of the client browser contacting the external site directly, the browser sends its request to the proxy (which is under the same origin or a controlled domain). The proxy then forwards the request to the external site on behalf of the client. When the external data comes back, the proxy adds the necessary Access-Control-Allow-Origin headers (for example, allowing www.ngene.org or using a wildcard *) before returning the response to the browser. By doing so, the response appears to the browser as coming from a permissive source, satisfying CORS requirements.

   In simpler terms, the CORS proxy (such as the service running at proxy.ngene.org) acts as a trusted middleman between the web crawler and target websites. It enables the main application to retrieve external resources that would otherwise be off-limits due to CORS. This approach is crucial for features like the nGene web crawler, which must gather information from various domains without being blocked by the same-origin policy.

3. Deployment Approaches for the Proxy Service

   Once the need for a proxy service (e.g., the CORS proxy) is identified, there are two principal ways to integrate it with the main site through Nginx. The service can either run under a separate subdomain as an independent virtual host, or it can be exposed via a specific path under the existing main domain. Each approach has implications for configuration, security, and complexity, as outlined below.

   1. Separate Subdomain (Independent Virtual Host)

      In this approach, the proxy runs on a distinct subdomain, for example proxy.ngene.org, separate from the main site’s domain. Nginx is configured with a dedicated server block for the subdomain, listening for requests to proxy.ngene.org. Those requests are then proxied to the internal service (such as a Docker container running the proxy logic) on its own port. Meanwhile, the main site www.ngene.org continues to operate under a different server block.

      Using a subdomain clearly delineates the proxy service. It can have its own access logs, performance settings, and even reside on a separate machine if necessary, without interfering with the main website’s configuration. This isolation also means that the proxy’s failures or restarts are less likely to impact the main site’s availability.

      One consideration, however, is that calls from the main site to this subdomain are cross-origin by nature. Since proxy.ngene.org is a different origin from www.ngene.org, web applications in the main domain must deal with CORS when communicating with the proxy. In practice, the CORS proxy service itself is built to handle this: it includes headers in its responses to permit the main site’s requests. Nonetheless, developers must ensure these headers are correctly configured (either in the proxy service’s code or via Nginx) so that browsers allow the interaction.

      Additional setup for a subdomain includes DNS configuration (pointing proxy.ngene.org to the server) and obtaining an SSL/TLS certificate covering that subdomain (which could be a wildcard certificate for *.ngene.org or a specific certificate for the proxy). Once in place, users can access the proxy functionality through the subdomain URL. This approach is common for public APIs and microservices, as it cleanly separates them under different hostnames.

   2. Path-Based Routing on the Main Domain

      This approach keeps everything on the primary domain, exposing the proxy service as a path under www.ngene.org (for instance, www.ngene.org/proxy/ or a similar endpoint). Rather than a separate server block, Nginx uses a location directive within the main server configuration to route requests with a certain URL prefix to the proxy service’s internal port.

      With path-based routing, end users and client scripts see the proxy as part of the main website. For example, a request for https://www.ngene.org/proxy/example.com/data could be internally forwarded by Nginx to the proxy service, which then fetches data from https://example.com/data and returns it. Because the browser is making a request to www.ngene.org (the same origin as the page), it will not trigger a CORS check at all. In effect, this method sidesteps CORS issues by keeping interactions under one domain from the perspective of the client.

      Path routing simplifies certain aspects: there is no need for a separate subdomain DNS entry or SSL certificate, and cookies or authentication tokens can be shared across the site and proxy path if needed. On the other hand, the Nginx configuration becomes a bit more complex. Care must be taken to choose a unique path that does not clash with existing URLs on the site. Also, the proxy service must be aware (or Nginx must be configured) to handle the path prefix — often by stripping the /proxy/ prefix before forwarding the request so that the internal service receives the expected URL format.

      While this single-domain approach tightly integrates the proxy service with the main site, it can be very convenient for development and user experience. It is often used when the proxy is an internal component meant only to support the main application’s functionality, and when minimizing cross-domain complexity is a priority.

   Comparison of the Two Approaches:

   Criteria	Separate Subdomain	Path-Based (Single Domain)
   Configuration	Each service has its own Nginx server block (virtual host) and configuration.	A single Nginx server block handles both services with different URL path locations.
   Domain & URL	Service is accessed via a distinct hostname (e.g., proxy.ngene.org).	Service is accessed via a path on the main site (e.g., www.ngene.org/proxy/...).
   CORS Considerations	Requests from www.ngene.org to proxy.ngene.org are cross-origin; the proxy must send CORS headers to allow these requests.	Requests stay on www.ngene.org; no browser CORS checks occur for using the proxy.
   SSL Certificates	Requires an SSL certificate covering the subdomain (either a wildcard or separate certificate for proxy.ngene.org).	Uses the main domain’s SSL certificate; no additional certificates needed for the proxy path.
   Isolation & Scalability	High isolation: the proxy service can be scaled or moved independently (even to a different server) as long as DNS points to the right place.	Tighter coupling: the proxy runs through the same domain, but Nginx can still route to a separate server or container in the backend if required.
   Setup Complexity	Involves managing multiple host configurations and DNS entries; clear separation but more initial setup.	Single host configuration and DNS entry; simpler on the surface, but requires careful internal route configuration.

4. Conclusion and Recommendations

   Both approaches—using a separate subdomain and using a path on the main domain—are viable for integrating a proxy service via Nginx. The optimal choice depends on the project’s requirements and future plans. If complete separation of services and maximum flexibility in deployment are desired, the subdomain strategy may be the better choice. It provides clear boundaries and can simplify scaling or relocating the proxy component later. On the other hand, if ease of integration and avoiding cross-origin complications are top priorities, the path-based approach offers a convenient solution under a unified domain.

   In the context of Project nGene.org, implementing the CORS proxy allowed the web crawler functionality to operate within browser constraints. Either deployment method would achieve this goal. It is advisable to evaluate factors such as maintenance overhead, domain management, and security policies when deciding between the two designs. By leveraging Nginx’s flexibility, the project can start with one approach and transition to the other in the future if needs change. Ultimately, the combination of Nginx and containerization provides a robust infrastructure, ensuring that both www.ngene.org and its proxy service run securely and efficiently in tandem.

Written on November 15, 2025

Firefox Extension

Analysis of Video Stream Extraction

youtube_downloader-1.6.30

The provided JavaScript code represents a Firefox extension named youtube_downloader-1.6.30 designed to facilitate the downloading of YouTube videos by extracting their stream URLs and saving them as local video files. The extension accomplishes this through several key components and functions, each playing a pivotal role in the extraction and downloading process. The following sections delineate the relevant parts of the code responsible for achieving this functionality.

1. Event Handling and Messaging Infrastructure

The extension employs an event-driven architecture to manage communication between different parts of the extension, such as content scripts and background scripts. This is facilitated by the Events module:

var Events = (function () {
    // Initialization of callbacks, listeners, and event handling mechanisms
    chrome.runtime.onMessage.addListener(function (request, sender, sendResponse) {
        // Dispatching events based on message type
    });

    function _sendMessage(type, data, tab_id) {
        // Sending messages either to specific tabs or the runtime
    }

    function _addListener(type, cb) {
        // Adding event listeners for specific message types
    }

    // Other utility functions for event management

    return {
        sendMessage: _sendMessage,
        addListener: _addListener,
        dispatchEvent: _dispatchEvent,
        addEventListener: _addEventListener
    };
})();

This module is foundational for handling asynchronous operations, such as fetching video information and responding to user interactions within the extension's UI.

2. User Interface Integration

The extension injects a download button into YouTube's watch page, enabling users to initiate the download process. Key functions involved in this process include:

• injectToYouTubeWatchPage: Initiates the injection of the download button and menu into the YouTube interface.
• addYouTubeWatchPageDownloadButton: Creates and inserts the download button into the YouTube page's DOM.
• addYouTubeWatchPageMenuPanel: Adds a menu panel that lists available download options once the button is clicked.

These functions ensure that users have a seamless interface to interact with, allowing them to select and download desired video formats.

3. Fetching and Parsing Video Information

To extract downloadable video streams, the extension retrieves and parses YouTube's video information. This is primarily handled by the g_downloadManager module:

var g_downloadManager = (function () {
    var _jsPlayerCache = {};
    var _cache = {};

    function __loadLinksForVideo(idVideo, callback, errback) {
        // Constructs YouTube info URLs and fetches video information
        function getVideodata(vInfo) {
            // Parses the video information to extract stream URLs and formats
        }
    }

    function _getLinksForVideo(idVideo, tab_id) {
        // Retrieves cached links or initiates fetching if not cached
    }

    function _downloadSignatureDecoderAndDownloadLinks(href, callback, failback) {
        // Downloads and deciphers the signature required to access video streams
        function parsePageContent(html) {
            // Extracts necessary configuration and JavaScript player URL from the page
        }
    }

    function _getParamFromUrl(url, keyName) {
        // Utility function to extract parameters from URLs
    }

    function _getFileExt(mime) {
        // Determines file extension based on MIME type
    }

    function _getFileNameFromUrl(url) {
        // Extracts the video title from the URL for naming the downloaded file
    }

    function _getVideoNameFromUrl(url) {
        // Combines the title and extension to form the complete filename
    }

    var _downloadVideo = (function () {
        var _lastDownloadedUrls = [];

        return function (url) {
            if (_lastDownloadedUrls.indexOf(url) < 0) {
                _lastDownloadedUrls.push(url);

                setTimeout(function () {
                    // prevent second download of the same url
                    var indexEl = _lastDownloadedUrls.indexOf(url);
                    if (indexEl >= 0) {
                        _lastDownloadedUrls.splice(indexEl, 1);
                    }
                }, 5000);

                setTimeout(function () {
                    var params = {
                        "url": url,
                        "saveAs": true,
                        "method": "GET",
                        "conflictAction": "uniquify"
                    };
                    var filename = _getVideoNameFromUrl(url);
                    if (filename) {
                        params["filename"] = filename;
                    }
                    chrome.downloads.download(params, function() {
                        d_log(chrome.runtime.lastError);
                    });
                }, 300);
            }
        };
    })();

    return {
        getLinksForVideo: _getLinksForVideo,
        downloadSignatureDecoderAndDownloadLinks: _downloadSignatureDecoderAndDownloadLinks,
        downloadVideo: _downloadVideo
    }
})();

1. Fetching Video Information: The function __loadLinksForVideo constructs YouTube's video information URLs and retrieves data necessary for identifying available video streams.
2. Parsing Stream Maps: Within getVideodata, the extension parses the stream_map parameter to extract individual stream URLs, signatures, and format details.
3. Signature Deciphering: YouTube often obfuscates video stream URLs using signatures. The extension addresses this by downloading the signature decoder JavaScript from YouTube's player page and executing it to obtain valid signatures. This is handled by functions such as getSignatureDecoder, applyDecoder, and related signature deciphering utilities.

This module ensures that the extension can reliably generate valid URLs for downloading video streams despite YouTube's protective measures.

4. Signature Deciphering Mechanism

YouTube employs signature-based protection for its video streams, necessitating a deciphering mechanism to obtain valid download URLs. The extension includes a comprehensive method to handle this:

var ytHtml5SignatureDecipher = {
    readObfFunc: function(func, data) {
        // Parses and interprets obfuscated signature functions
    },
    getNewChip: function (data) {
        // Extracts transformation actions from YouTube's player code
    },
    getChip: function(data) {
        // Processes the player code to retrieve the necessary transformation steps
    }
};

var getDecodeSignatureFunc = function(data) {
    var actList = ytHtml5SignatureDecipher.getChip(data);
    return function (s) {
        return getSig(actList, s);
    }
}

var getTransformUrlFunc = function(data) {
    var nTransformFunc = getNTransformFunc(data);
    return function (url) {
        var n_param = url.match(/&n=([^&]+)&/);
        if (!n_param) {
            return url;
        }
        n_param = n_param[1];
        return url.replace(n_param, nTransformFunc(n_param));
    }
}

• ytHtml5SignatureDecipher: Contains methods to analyze and interpret YouTube's obfuscated JavaScript functions responsible for signature generation.
• getDecodeSignatureFunc: Utilizes the deciphering logic to create a function that can decode the signature present in stream URLs.
• getTransformUrlFunc: Handles additional URL transformations required to access the video streams correctly.

This mechanism ensures that the extension can reliably generate valid URLs for downloading video streams despite YouTube's protective measures.

5. Extracting and Organizing Download Links

Once the video information and signatures are deciphered, the extension organizes the available download links for user selection:

function getLinksFromFormatListAndStreamMap(fmt_list, fmt_stream_map) {
    // Parses format lists and stream maps to extract downloadable URLs
}

function getLinksFromStreamingDataFormats(formats) {
    // Processes streaming data formats to obtain download links
}

function newYouTubeDownloadLink(url, sign, type, resolution, quality, stereo3d, itag) {
    // Constructs a structured object representing a downloadable video link
}

function removeYouTubeDownloadLinksDuplicates(originalLinks) {
    // Filters out duplicate download links to ensure uniqueness
}

1. Parsing Formats: Functions like getLinksFromFormatListAndStreamMap and getLinksFromStreamingDataFormats parse YouTube's provided format lists to identify available video streams along with their respective qualities and types.
2. Organizing Links: The newYouTubeDownloadLink function structures each stream's information, including URL, type, resolution, quality, and other relevant metadata.
3. Eliminating Duplicates: To present a clean list of options to the user, removeYouTubeDownloadLinksDuplicates ensures that each download link is unique, preventing redundant entries.

6. Initiating Downloads

Upon user selection, the extension initiates the download process using Chrome's downloads API:

var _downloadVideo = (function () {
    var _lastDownloadedUrls = [];

    return function (url) {
        if (_lastDownloadedUrls.indexOf(url) < 0) {
            _lastDownloadedUrls.push(url);

            setTimeout(function () {
                // prevent second download of the same url
                var indexEl = _lastDownloadedUrls.indexOf(url);
                if (indexEl >= 0) {
                    _lastDownloadedUrls.splice(indexEl, 1);
                }
            }, 5000);

            setTimeout(function () {
                var params = {
                    "url": url,
                    "saveAs": true,
                    "method": "GET",
                    "conflictAction": "uniquify"
                };
                var filename = _getVideoNameFromUrl(url);
                if (filename) {
                    params["filename"] = filename;
                }
                chrome.downloads.download(params, function() {
                    d_log(chrome.runtime.lastError);
                });
            }, 300);
        }
    };
})();

• URL Validation: The function ensures that the same URL is not downloaded multiple times within a short timeframe, enhancing efficiency and preventing duplicate downloads.
• Filename Determination: It extracts a suitable filename from the URL parameters, ensuring that the downloaded file is appropriately named and formatted.
• Download Execution: Utilizing chrome.downloads.download, the extension triggers the download, allowing the user to save the video file locally.

7. Integration and Initialization

Finally, the extension initializes its components and sets up necessary event listeners to ensure seamless operation:

function main() {
    function openTab(ignore, data) {
        if (typeof data.url !== "undefined") {
            g_downloadManager.downloadVideo(data.url);
        }
    }

    Events.addListener("openBYDTab", openTab);

    Events.addListener("get_links", function (type, idVideo, ignore, tab_id) {
        g_downloadManager.getLinksForVideo(idVideo, tab_id);
    });

    Events.addEventListener("get_sig_decoder", function (event) {
        var data = event.data;
        if (data && data.href) {
            g_downloadManager.downloadSignatureDecoderAndDownloadLinks(data.href, function (data) {
                event.reply({
                    res: true,
                    data: data
                });
            }, function () {
                event.reply({
                    res: false
                });
            });
        }
    });
}

main();

• Event Listeners: The extension listens for specific events such as openBYDTab, get_links, and get_sig_decoder to manage user interactions and data processing dynamically.
• Initialization: By invoking main(), the extension sets up its operational framework, ensuring that all components are ready to handle tasks as users interact with the download features.

Written on December 3rd, 2024

easy_youtube_video_download-19.1

The provided JavaScript code represents a Firefox extension named easy_youtube_video_download-19.1 designed to facilitate the downloading of YouTube videos by extracting their stream URLs and saving them as local video files. The extension achieves this through several key components and functions, each contributing to the extraction and downloading process. The following sections identify and elaborate on the relevant parts of the code responsible for accomplishing this functionality.

1. Event Handling and Options Management

The extension manages user preferences and handles various installation events through dedicated functions and event listeners. This ensures that user settings are preserved and appropriate actions are taken during installation, updates, or uninstallation.

a. Saving and Restoring Options

The functions save_options and restore_options are responsible for handling user preferences related to autoplay, premium keys, and notification settings. These functions interact with Chrome's storage API to persist and retrieve user settings.

function save_options(e) {
    e.preventDefault();
    var autop = document.getElementById('autoplay').checked;
    var pKey = document.getElementById('prokey').value;
    var noNotify = document.getElementById('notification').checked;
    chrome.storage.sync.set({
        autop: autop,
        pKey: pKey,
        noNotify: noNotify
    }, function () {
        // Update status to let user know options were saved.
        var status = document.getElementById('status');
        status.textContent = 'Options saved.';
        setTimeout(function () {
            status.textContent = '';
        }, 1750);
    });
}

// Restores select box and checkbox state using the preferences stored in chrome.storage.
function restore_options() {
    // Use default value 
    chrome.storage.sync.get({
        autop: false,
        pKey: "",
        noNotify: false,
    }, function (items) {
        document.getElementById('autoplay').checked = items.autop;
        document.getElementById('prokey').value = items.pKey;
        document.getElementById('notification').checked = items.noNotify;
    });

    //do other common check tasks
    bootstart();
}
document.addEventListener('DOMContentLoaded', restore_options);
document.querySelector("form").addEventListener("submit", save_options);

b. Installation and Uninstallation Events

The extension listens for installation, update, and uninstallation events to provide appropriate user feedback and handle cleanup tasks.

//CHECK INSTALL, UPDATE, UNINSTALL
chrome.runtime.onInstalled.addListener(function (details) {
    if (details.reason == "install") {
        chrome.tabs.create({
            url: "https://www.yourvideofile.org/install-success.html",
        });
    }

    if (details.reason == "update") {
        chrome.tabs.create({
            url: "https://www.yourvideofile.org/update-success.html",
        });
    }
});

chrome.runtime.setUninstallURL(
    "https://www.yourvideofile.org/uninstall-success.html"
);

These listeners ensure that users receive confirmations upon installing, updating, or uninstalling the extension, enhancing user experience and providing necessary information.

2. User Interface Integration

The extension integrates seamlessly with YouTube's interface by injecting a download button and corresponding menu into the video watch page. This allows users to initiate the download process directly from the YouTube page.

a. Injecting the Download Button

The parseDetails function is pivotal in injecting the download button into the YouTube page. It determines the appropriate location within the DOM to place the button based on the YouTube UI version.

let dButton = document.createElement("button");
dButton.setAttribute("id", "ytdl_btn");
dButton.setAttribute("class", "ytdl_btn");
dButton.textContent = " " + buttonText + ": ▼" + " ";
dButton.setAttribute("data-tooltip-text", buttonLabel);

// Check if the extension is on the old or new YouTube UI
let isOldUI = true;
if (document.getElementById("comment-teaser")) {
    isOldUI = false;
}

if (parentElement && isOldUI) {
    //OLD UI
    parentElement.childNodes[0].appendChild(dButton);
}

//Are we on a new design, generate and add new style button
if (parentElement && !isOldUI) {
    console.log("Inside new UI");
    parentElement = document.getElementById("owner");
    parentElement.appendChild(dButton);
    buttonEle = document.getElementById("ytdl_btn");
    buttonEle.setAttribute("style", "border-radius: 30px;");
}

if (!parentElement) {
    // Handle cases where the button could not be attached to the standard locations
    let msgCont =
        "
Oops, unable to attach button to the original location on the page - this seems to be some code/layout change by Youtube, for the time-being you can use the button below. Once this design is finalised and pushed to all by Youtube, an updated addon version will place the button to usual location.

You can read more about this here.

";
    showPopup("", msgCont, true);
    parentElement = document.getElementById("notificationPopup");
    parentElement.childNodes[2].appendChild(dButton);
}

b. Creating the Download Menu

The extension generates a download menu containing various format options based on the extracted video streams. This menu is appended to the download button, allowing users to select their preferred download format.

// create download list
let dList = document.createElement("div");
dList.setAttribute("id", "ytdl_list");
dList.setAttribute("status", "hide");
dList.classList.add("ytdl_list", "ytdl_list_hide");
dButton.appendChild(dList);

for (let i = 0; i < downloadCodeList.length; i++) {
    let getF = downloadCodeList[i].format;
    if (FORMAT_LABEL[getF]) {
        let linkDiv = document.createElement("div");
        linkDiv.setAttribute("class", "eytd_list_item");

        let dLink = document.createElement("a");
        let url = DOMPurify.sanitize(downloadCodeList[i].url);
        dLink.setAttribute("id", "ytdl_link_" + downloadCodeList[i].format);
        dLink.setAttribute("loop", i + "");
        dLink.innerText = downloadCodeList[i].label;

        // Handle direct and external links
        if (downloadCodeList[i].download || downloadCodeList[i].external) {
            dLink.setAttribute("href", url);
            if (downloadCodeList[i].label != "Settings") {
                dLink.setAttribute("download", downloadCodeList[i].download);
            }
            dLink.setAttribute("target", "_blank");
            if (!downloadCodeList[i].external) {
                dLink.addEventListener("click", notifyExtension, false);
            }
        } else {
            live("click", "ytdl_link_" + downloadCodeList[i].format, function () {
                var frm_div = document.getElementById("EXT_DIV");
                if (frm_div) {
                    frm_div.parentElement.removeChild(frm_div);
                }
                var mp3_clean_url =
                    "https://videodroid.org/v3/authenticate.php?vid=" +
                    videoId +
                    "&stoken=" +
                    proKey +
                    "&format=" +
                    FORMAT_LABEL[getF] +
                    "&title=" +
                    videoTitle +
                    "&ver=" +
                    version;
                mp3_clean_url = encodeURI(mp3_clean_url);
                addiframe(mp3_clean_url, "250"); //210
                return false;
            });
        }

        linkDiv.appendChild(dLink);
        dList.appendChild(linkDiv);
    }
}

var downloadBtn = document.getElementById("ytdl_btn");
downloadBtn.addEventListener("click", expandList);

This section ensures that users can interact with the download options, selecting the desired video format for download directly from the YouTube interface.

3. Fetching and Parsing Video Information

Extracting downloadable video streams necessitates retrieving and parsing detailed video information from YouTube. This process involves interacting with YouTube's APIs and handling various data structures to obtain the necessary stream URLs and formats.

a. Fetching Video Data

The functions getRawPageData and getInnerApijson are responsible for fetching raw video data from YouTube. They interact with YouTube's internal APIs to obtain streaming information necessary for constructing download links.

async function getRawPageData() {
    injectScript(
        "var storage=window.localStorage;const videoPage = window?.ytplayer?.config?.args?.raw_player_response;storage.setItem('videoPage',JSON.stringify(videoPage));const $ = (s, x = document) => x.querySelector(s);const basejs =(typeof ytplayer !== 'undefined' && 'config' in ytplayer && ytplayer.config.assets? 'https://' + location.host + ytplayer.config.assets.js: 'web_player_context_config' in ytplayer? 'https://' + location.host + ytplayer.web_player_context_config.jsUrl: null) || $('script[src$=\"base.js\"]').src;storage.setItem('basejs',basejs);"
    );

    let videoPage = window.localStorage.getItem("videoPage");
    console.log("Fetched Raw Page Data:" + videoPage);
    return videoPage;
}

async function getInnerApijson(videoId, clientName, isAgeRestricted) {
    // Define client configurations with apiKey defined separately
    const clients = {
        "IOS_CREATOR": {
            clientDetails: {
                clientName: "IOS_CREATOR",
                clientVersion: "22.33.101",
                deviceModel: "iPhone14,3",
                userAgent: "com.google.ios.ytcreator/22.33.101 (iPhone14,3; U; CPU iOS 15_6 like Mac OS X)",
                hl: "en",
                timeZone: "UTC",
                utcOffsetMinutes: 0
            },
            apiKey: "AIzaSyA8eiZmM1FaDVjRy-dfKTyQ_vz_yYM39w"  // API key for IOS_CREATOR
        },
        "WEB": {
            clientDetails: {
                clientName: "WEB",
                clientVersion: "2.20201021.00.00",
                deviceModel: "",
                userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36",
                hl: "en",
                timeZone: "UTC",
                utcOffsetMinutes: 0
            },
            apiKey: "AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8"  // API key for WEB
        },
        "IOS": {
            clientDetails: {
                "clientName": "IOS",
                "clientVersion": "19.09.3",
                "deviceModel": "iPhone14,3",
                "userAgent": "com.google.ios.youtube/19.09.3 (iPhone14,3; U; CPU iOS 15_6 like Mac OS X)",
                "hl": "en",
                "timeZone": "UTC",
                "utcOffsetMinutes": 0
            },
            apiKey: "AIzaSyB-63vPrdThhKuerbB2N_l7Kwwcxj6yUAc"  // API key for IOS
        }
    };

    // Select the appropriate client configuration based on the input
    const { clientDetails, apiKey } = clients[clientName] || clients["IOS_CREATOR"]; // Default to IOS_CREATOR if clientName is not found

    // Customize client information based on age restriction
    const clientInfo = { ...clientDetails };
    if (isAgeRestricted) {
        clientInfo.clientVersion = clientDetails.clientVersion + "_restricted"; // Example suffix for age-restricted content
        clientInfo.clientScreen = "EMBED"; // This detail may be adjusted based on your needs
    }

    // Construct the request body
    const requestBody = {
        context: { client: clientInfo },
        videoId: videoId,
        playbackContext: {
            contentPlaybackContext: {
                html5Preference: "HTML5_PREF_WANTS"
            }
        },
        contentCheckOk: true,
        racyCheckOk: true
    };

    // Define the fetch request details with the updated body structure
    const requestOptions = {
        method: "post",
        headers: {
            Accept: "application/json, text/plain, */*",
            "Content-Type": "application/json",
        },
        body: JSON.stringify(requestBody)
    };

    // Construct the fetch URL using the client's API key
    const url = `https://youtubei.googleapis.com/youtubei/v1/player?key=${apiKey}`;

    try {
        // Execute the fetch request
        const response = await fetch(url, requestOptions);
        if (!response.ok) {
            throw new Error(`HTTP error! Status: ${response.status}`);
        }
        return await response.json();
    } catch (error) {
        console.error("Failed to fetch video page:", error);
        return null; // Optionally return an error object or handle error differently
    }
}

b. Parsing and Deciphering Signatures

YouTube employs signature-based protection to secure its video streams. The extension includes mechanisms to parse and decipher these signatures, enabling the construction of valid download URLs.

const parseDecsig = (data) => {
    try {
        const fnnameresult = /=([a-zA-Z0-9\$]+?)\(decodeURIComponent/.exec(data);
        const fnname = fnnameresult[1];
        const _argnamefnbodyresult = new RegExp(
            escapeRegExp(fnname) + "=function\\((.+?)\\){((.+)=\\2.+?)}"
        ).exec(data);
        const [_, argname, fnbody] = _argnamefnbodyresult;
        const helpernameresult = /;(.+?)\..+?\(/.exec(fnbody);
        const helpername = helpernameresult[1];
        const helperresult = new RegExp(
            "var " + escapeRegExp(helpername) + "={[\\s\\S]+?};"
        ).exec(data);
        const helper = helperresult[0];
        console.log(`parsedecsig result: ${argname}=>{${helper}\n${fnbody}}`);
        return new Function([argname], helper + "\n" + fnbody);
    } catch (e) {
        console.error("parsedecsig error:", e);
        console.info("script content:", data);
        console.info(
            'If you encounter this error, please copy the full "script content" to https://pastebin.com/ for me.'
        );
    }
};

This function analyzes YouTube's obfuscated JavaScript to extract and interpret the signature deciphering logic. By reconstructing the necessary functions, the extension can decode the signatures appended to video stream URLs, ensuring that the download links are valid and accessible.

4. Extracting and Organizing Download Links

After fetching and deciphering the necessary video information and signatures, the extension processes this data to generate a list of downloadable video streams in various formats and qualities.

a. Processing Video Formats

The functions displayFMT and parseDetails are central to organizing the available video formats into user-friendly download options.

async function displayFMT(finalFmt, videoTitle) {
    let jsonData = {};
    let downloadCodeList = [];

    const fmtMap1 = finalFmt.map((format) => {
        jsonData[format.itag] = format._decryptedURL;

        let fmt_url = format._decryptedURL;

        if (fmt_url != undefined && FORMAT_LABEL[format.itag] != undefined) {
            downloadCodeList.push({
                url: DOMPurify.sanitize(fmt_url),
                format: format.itag,
                label: FORMAT_LABEL[format.itag],
                download: videoTitle + "." + FORMAT_TYPE[format.itag],
            });
        }

        //Check 720p Dash availability
        if (format.itag == "22") {
            is720p = true;
        }

        if (format.itag == "136" || format.itag == "247") {
            is720pDash = true;
        }

        //Check 1080p availability
        if (format.itag == "137" || format.itag == "299" || format.itag == "303" || format.itag == "335" || format.itag == "617" || format.itag == "636") {
            is1080p = true;
        }
    });

    //Add additional buttons to the list
    //720p
    if (is720pDash && !is720p) {
        downloadCodeList.push({
            url: DOMPurify.sanitize("https://videodroid.org/"),
            format: "720P",
            label: "MP4 720p (HD)",
        });
    }

    //Full-HD
    if (is1080p) {
        downloadCodeList.push({
            url: DOMPurify.sanitize("https://videodroid.org/"),
            format: "1080p3",
            label: "Full-HD 1080p",
        });
        //reset the value just in case
        is1080p = false;
    }

    downloadCodeList.push({
        url: DOMPurify.sanitize("https://videodroid.org/"),
        format: "mp3256",
        label: "MP3 HQ (256 Kbps)",
    });
    downloadCodeList.push({
        url: DOMPurify.sanitize("https://videodroid.org/"),
        format: "mp3128",
        label: "MP3 HQ (128 Kbps)",
    });

    //Options
    let lnk = browser.runtime.getURL("options/options.html");
    downloadCodeList.push({
        url: lnk,
        format: "Settings",
        label: "Settings",
        external: true,
    });
    //About-Help
    downloadCodeList.push({
        url: DOMPurify.sanitize(
            "https://www.yourvideofile.org/support.html?&ver=" + version
        ),
        format: "About",
        label: "Contact/Bug Report",
        external: true,
    });

    //Donation
    downloadCodeList.push({
        url: DOMPurify.sanitize(
            "https://videodroid.org/pro_upgrade.html?&ver=" + version
        ),
        format: "Donate",
        label: "Donate",
        external: true,
    });

    return downloadCodeList;
}

The displayFMT function processes the final list of video formats, sanitizes the URLs using DOMPurify, and organizes them into a structured list of download options. This list includes various video qualities and formats, as well as additional functionalities such as settings, contact, and donation links.

b. Parsing Video Details

The parseDetails function orchestrates the extraction process by fetching video data, handling potential age restrictions, deciphering signatures, and preparing the download options.

async function parseDetails(url) {
    if (window.location.href.indexOf("shorts/") > -1) {
        let msgCont = "Youtube Shorts video uses a different UI and to download these videos you can use the Download menu provided via the Browser toolbar button.";
        showPopup("", msgCont, true);
        videoId = window.location.href.split("shorts/")[1];
    } else {
        const query = parseQueryString(url.split("?")[1]);
        videoId = query["v"];
    };

    let videoPage = await getInnerApijson(videoId, "IOS_CREATOR", false);

    if (!videoPage.streamingData) {
        //Seems age gated video, refetch data
        videoPage = await getInnerApijson(videoId, "IOS_CREATOR", true);
        console.log("Using Age gated Android");
    }

    //If it still fails, try using page data
    if (!videoPage.streamingData.formats) {
        videoPage = await getRawPageData();
        videoPage = JSON.parse(videoPage);

        //save decsig function for usage later
        var basejs = window.localStorage.getItem("basejs");
        console.log("Scraping page data, and getting base.js : " + basejs)
        var decsig = await fetch(basejs)
            .then((res) => res.text())
            .then((body) => {
                return body;
            });

        var decsig = await parseDecsig(decsig);
        console.log("Youtube Code Changed API Failed");
    }

    //we still have a failure, display error
    if (!videoPage.streamingData.formats) {
        let msgCont = "Error!!";
        //this could be a live video check and inform
        if (
            videoPage.videoDetails.isLive ||
            videoPage.playabilityStatus.reason == "This live event has ended."
        ) {
            msgCont =
                "
This is either an ongoing or recently finished Live stream, it can take upto 12-72 hours to generate download links for such videos, pls. try later after 12-72 hours and the links should be availble by then.
";
            showPopup("", msgCont, true);
        } else {
            msgCont =
                "
We were not able to parse the download links from this page, try a page refresh. If this happens with all videos do report this to us.";
            showPopup("", msgCont, true);
        }
    }

    let videoTitle = document.title
        .replace(/^\(\d+\)\s*/, "")
        .replace(/\s*\-\s*YouTube$|'/g, "")
        .replace(/^\s+|\s+$|\.+$/g, "")
        .replace(/[\\/:"*?<>|]/g, "")
        .replace(/[\x00-\x1f\x7f]/g, "")
        .replace(/[\|\\\/]/g, window.navigator.userAgent.indexOf("Win") >= 0 || window.navigator.userAgent.indexOf("Mac") >= 0 ? "-" : "")
        .replace(/^(con|prn|aux|nul|com\d|lpt\d)$/i, "")
        .replace(/#/g, window.navigator.userAgent.indexOf("Win") >= 0 ? "" : "%23")
        .replace(/&/g, window.navigator.userAgent.indexOf("Win") >= 0 ? "_" : "%26");

    const formatURLs = videoPage.streamingData.formats.map((format) => {
        let url = format.url;
        const cipher = format.signatureCipher || format.cipher;

        if (!!cipher) {
            const components = parseQueryString(cipher);
            const sig = decsig(components.s);
            url =
                components.url +
                `&${encodeURIComponent(components.sp)}=${encodeURIComponent(sig)}`;
        }

        return {
            itag: format.itag,
            _decryptedURL: url,
        };
    });

    //Populate Adaptive
    let adaptiveFormats;
    if (videoPage.streamingData.adaptiveFormats) {
        adaptiveFormats = videoPage.streamingData.adaptiveFormats;
    } else {
        // Fetch using another Client if adaptiveFormats are not available
        videoPage = await getInnerApijson(videoId, "IOS", false);
        adaptiveFormats = videoPage.streamingData.adaptiveFormats ? videoPage.streamingData.adaptiveFormats : [];
    }
    const adaptiveFormatURLs = videoPage.streamingData.adaptiveFormats.map(
        (format) => {
            let url = format.url;
            const cipher = format.signatureCipher || format.cipher;

            if (!!cipher) {
                const components = parseQueryString(cipher);
                const sig = decsig(components.s);
                url =
                    components.url +
                    `&${encodeURIComponent(components.sp)}=${encodeURIComponent(sig)}`;
            }

            return {
                itag: format.itag,
                _decryptedURL: url,
            };
        }
    );

    const finalFmt = [...formatURLs, ...adaptiveFormatURLs];
    var downloadCodeList = await displayFMT(finalFmt, videoTitle);

    let data = { VideoData: downloadCodeList, videoTitle: videoTitle, key: proKey };
    sessionStorage.setItem("dList_" + videoId, JSON.stringify(data));

    // Prepare button text based on language settings
    var language = document.documentElement.getAttribute("lang");
    language = language.substring(0, 2);
    var buttonText = BUTTON_TEXT[language]
        ? BUTTON_TEXT[language]
        : BUTTON_TEXT["en"];
    var buttonLabel = BUTTON_TOOLTIP[language]
        ? BUTTON_TOOLTIP[language]
        : BUTTON_TOOLTIP["en"];

    // Inject the download button into the YouTube page
    let dButton = document.createElement("button");
    dButton.setAttribute("id", "ytdl_btn");
    dButton.setAttribute("class", "ytdl_btn");
    dButton.textContent = " " + buttonText + ": ▼" + " ";
    dButton.setAttribute("data-tooltip-text", buttonLabel);

    // Additional UI handling omitted for brevity
}

The parseDetails function coordinates the overall extraction process by:

• Identifying the video ID from the URL.
• Fetching video data, including handling age-restricted content.
• Deciphering signatures to construct valid download URLs.
• Generating a list of downloadable formats and injecting the download button into the YouTube page.

4. Signature Deciphering Mechanism

To bypass YouTube's signature-based protection on video streams, the extension includes a signature deciphering mechanism. This involves analyzing YouTube's obfuscated JavaScript to reconstruct the functions necessary for decoding signatures.

a. Deciphering Function Extraction

The parseDecsig function parses the signature deciphering logic from YouTube's JavaScript code. It identifies and reconstructs the necessary functions to decode the signatures appended to video stream URLs.

const parseDecsig = (data) => {
    try {
        const fnnameresult = /=([a-zA-Z0-9\$]+?)\(decodeURIComponent/.exec(data);
        const fnname = fnnameresult[1];
        const _argnamefnbodyresult = new RegExp(
            escapeRegExp(fnname) + "=function\\((.+?)\\){((.+)=\\2.+?)}"
        ).exec(data);
        const [_, argname, fnbody] = _argnamefnbodyresult;
        const helpernameresult = /;(.+?)\..+?\(/.exec(fnbody);
        const helpername = helpernameresult[1];
        const helperresult = new RegExp(
            "var " + escapeRegExp(helpername) + "={[\\s\\S]+?};"
        ).exec(data);
        const helper = helperresult[0];
        console.log(`parsedecsig result: ${argname}=>{${helper}\n${fnbody}}`);
        return new Function([argname], helper + "\n" + fnbody);
    } catch (e) {
        console.error("parsedecsig error:", e);
        console.info("script content:", data);
        console.info(
            'If you encounter this error, please copy the full "script content" to https://pastebin.com/ for me.'
        );
    }
};

By reconstructing the signature deciphering function, the extension ensures that it can generate valid signatures required to access and download video streams.

b. Applying the Deciphered Signature

Once the deciphering function is obtained, it is applied to the encrypted signatures to generate valid URLs for downloading the video streams.

const formatURLs = videoPage.streamingData.formats.map((format) => {
    let url = format.url;
    const cipher = format.signatureCipher || format.cipher;

    if (!!cipher) {
        const components = parseQueryString(cipher);
        const sig = decsig(components.s);
        url =
            components.url +
            `&${encodeURIComponent(components.sp)}=${encodeURIComponent(sig)}`;
    }

    return {
        itag: format.itag,
        _decryptedURL: url,
    };
});

This section ensures that each video stream URL is properly constructed with a valid signature, making the download links accessible and functional.

5. Initiating Downloads

Upon user selection of a desired video format, the extension initiates the download process using Chrome's downloads API. This allows the user to save the video file locally in the chosen format and quality.

a. Handling Download Requests

The extension listens for download requests and processes them accordingly. It sanitizes the filenames to prevent illegal characters and ensures that downloads are not duplicated.

chrome.runtime.onMessage.addListener(function (message) {
    let fname = message.filename
        .trim()
        .replace(/[~!@#$%^&*()_|+\-=?;:'",<>{}[\]\\/]/gi, "-")
        .replace(/[\\/:*?"<>|]/g, "_")
        .substring(0, 240)
        .replace(/\s+/g, " ");

    chrome.downloads.download({
        url: message.url,
        filename: fname,
        conflictAction: "uniquify"
    }, function (downloadId) {
        if (typeof downloadId !== 'undefined') {
            console.log('Download initiated, ID is: ' + downloadId);
        } else {
            console.error('EYTD Download : ' + chrome.runtime.lastError.message);
            alert(chrome.runtime.lastError.message + ", This could be due to illegal characters in video title, try right-click and 'Save Link As' to download.")
        }
    });
});

This listener ensures that download requests are handled efficiently, providing feedback on the download status and alerting the user in case of errors.

b. Download Management

The function _downloadVideo within the g_downloadManager module manages the actual download process. It validates URLs, determines appropriate filenames, and utilizes Chrome's downloads API to initiate the download.

var _downloadVideo = (function () {
    var _lastDownloadedUrls = [];

    return function (url) {
        if (_lastDownloadedUrls.indexOf(url) < 0) {
            _lastDownloadedUrls.push(url);

            setTimeout(function () {
                // prevent second download of the same url
                var indexEl = _lastDownloadedUrls.indexOf(url);
                if (indexEl >= 0) {
                    _lastDownloadedUrls.splice(indexEl, 1);
                }
            }, 5000);

            setTimeout(function () {
                var params = {
                    "url": url,
                    "saveAs": true,
                    "method": "GET",
                    "conflictAction": "uniquify"
                };
                var filename = _getVideoNameFromUrl(url);
                if (filename) {
                    params["filename"] = filename;
                }
                chrome.downloads.download(params, function() {
                    d_log(chrome.runtime.lastError);
                });
            }, 300);
        }
    };
})();

This mechanism ensures that downloads are initiated smoothly while preventing duplicate downloads within a short timeframe, thereby enhancing the extension's reliability and user experience.

6. Fetching and Deciphering Video Streams

The extension employs sophisticated methods to fetch video stream data and decipher the necessary signatures to construct valid download URLs. This involves interacting with YouTube's internal APIs and handling various data formats.

a. Fetching Video Stream Data

The function getInnerApijson interacts with YouTube's internal APIs to retrieve detailed streaming data for a given video ID. It handles different client configurations and accommodates age-restricted content.

async function getInnerApijson(videoId, clientName, isAgeRestricted) {
    // Define client configurations with apiKey defined separately
    const clients = {
        "IOS_CREATOR": {
            clientDetails: {
                clientName: "IOS_CREATOR",
                clientVersion: "22.33.101",
                deviceModel: "iPhone14,3",
                userAgent: "com.google.ios.ytcreator/22.33.101 (iPhone14,3; U; CPU iOS 15_6 like Mac OS X)",
                hl: "en",
                timeZone: "UTC",
                utcOffsetMinutes: 0
            },
            apiKey: "AIzaSyA8eiZmM1FaDVjRy-dfKTyQ_vz_yYM39w"  // API key for IOS_CREATOR
        },
        // Additional client configurations omitted for brevity
    };

    // Select the appropriate client configuration based on the input
    const { clientDetails, apiKey } = clients[clientName] || clients["IOS_CREATOR"]; // Default to IOS_CREATOR if clientName is not found

    // Customize client information based on age restriction
    const clientInfo = { ...clientDetails };
    if (isAgeRestricted) {
        clientInfo.clientVersion = clientDetails.clientVersion + "_restricted"; // Example suffix for age-restricted content
        clientInfo.clientScreen = "EMBED"; // This detail may be adjusted based on your needs
    }

    // Construct the request body
    const requestBody = {
        context: { client: clientInfo },
        videoId: videoId,
        playbackContext: {
            contentPlaybackContext: {
                html5Preference: "HTML5_PREF_WANTS"
            }
        },
        contentCheckOk: true,
        racyCheckOk: true
    };

    // Define the fetch request details with the updated body structure
    const requestOptions = {
        method: "post",
        headers: {
            Accept: "application/json, text/plain, */*",
            "Content-Type": "application/json",
        },
        body: JSON.stringify(requestBody)
    };

    // Construct the fetch URL using the client's API key
    const url = `https://youtubei.googleapis.com/youtubei/v1/player?key=${apiKey}`;

    try {
        // Execute the fetch request
        const response = await fetch(url, requestOptions);
        if (!response.ok) {
            throw new Error(`HTTP error! Status: ${response.status}`);
        }
        return await response.json();
    } catch (error) {
        console.error("Failed to fetch video page:", error);
        return null; // Optionally return an error object or handle error differently
    }
}

This function ensures that the extension can retrieve comprehensive streaming data necessary for constructing download links, even accommodating scenarios involving age restrictions.

b. Deciphering and Constructing Download URLs

After obtaining the streaming data, the extension deciphers any encrypted signatures and constructs valid download URLs for each available video format.

const formatURLs = videoPage.streamingData.formats.map((format) => {
    let url = format.url;
    const cipher = format.signatureCipher || format.cipher;

    if (!!cipher) {
        const components = parseQueryString(cipher);
        const sig = decsig(components.s);
        url =
            components.url +
            `&${encodeURIComponent(components.sp)}=${encodeURIComponent(sig)}`;
    }

    return {
        itag: format.itag,
        _decryptedURL: url,
    };
});

// Similar processing for adaptive formats omitted for brevity

This mapping ensures that each video stream URL is properly decrypted and sanitized, making them ready for user-initiated downloads.

7. Security and Sanitization

To maintain security and prevent potential vulnerabilities, the extension employs sanitization mechanisms to cleanse URLs and user inputs. This is crucial in mitigating risks such as cross-site scripting (XSS).

a. Sanitizing URLs

The extension uses DOMPurify to sanitize URLs before embedding them into the DOM or initiating downloads. This ensures that only safe and valid URLs are processed.

let url = DOMPurify.sanitize(downloadCodeList[i].url);
dLink.setAttribute("href", url);

b. Validating User Inputs

The function isValidEmail validates email inputs to ensure that only correctly formatted emails are accepted, enhancing the integrity of user-provided data.

function isValidEmail(email) {
    const re = /^(([^<>()\[\]\\.,;:\s@"]+(\.[^<>()\[\]\\.,;:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/;
    return re.test(String(email).toLowerCase());
}

By validating and sanitizing inputs, the extension upholds high security standards, protecting both the user and the system from malicious exploits.

8. Additional Features and Enhancements

The extension incorporates additional features to enhance user experience and provide extended functionalities beyond basic downloading capabilities.

a. File Size Calculation

The function addFileSize calculates and displays the size of each downloadable file, providing users with information about the download before initiating it.

function addFileSize(url, format) {
    function updateVideoLabel(size, format) {
        var elem = document.getElementById("ytdl_link_" + format);
        if (elem) {
            size = parseInt(size, 10);
            if (size >= 1073741824) {
                size = parseFloat((size / 1073741824).toFixed(1)) + " GB";
            } else if (size >= 1048576) {
                size = parseFloat((size / 1048576).toFixed(1)) + " MB";
            } else {
                size = parseFloat((size / 1024).toFixed(1)) + " KB";
            }
            if (elem.childNodes.length > 1) {
                elem.lastChild.nodeValue = " (" + size + ")";
            } else if (elem.childNodes.length == 1) {
                elem.appendChild(document.createTextNode(" (" + size + ")"));
            }
        }
    }
    let matchSize = findMatch(url, /[&\?]clen=([0-9]+)&/i);
    if (matchSize) {
        updateVideoLabel(matchSize, format);
    } else {
        if (url.indexOf("googlevideo.com") !== -1) {
            fetch(url, {
                method: "HEAD",
            })
                .then((response) => {
                    let size = response.headers.get("content-length");
                    if (size) {
                        updateVideoLabel(size, format);
                    }
                })
                .catch((error) => {
                    console.log("Error Fetch Filesize URL : " + error);
                });
        }
    }
}

b. Dynamic UI Adjustments

The extension dynamically adjusts the user interface based on the YouTube page's structure, ensuring compatibility with different YouTube layouts and versions.

function parseDetails(url) {
    // ... [Code omitted for brevity]

    // Inject the download button into the YouTube page
    let dButton = document.createElement("button");
    dButton.setAttribute("id", "ytdl_btn");
    dButton.setAttribute("class", "ytdl_btn");
    dButton.textContent = " " + buttonText + ": ▼" + " ";
    dButton.setAttribute("data-tooltip-text", buttonLabel);

    // Check if the extension is on the old or new YouTube UI
    let isOldUI = true;
    if (document.getElementById("comment-teaser")) {
        isOldUI = false;
    }

    if (parentElement && isOldUI) {
        //OLD UI
        parentElement.childNodes[0].appendChild(dButton);
    }

    //Are we on a new design, generate and add new style button
    if (parentElement && !isOldUI) {
        console.log("Inside new UI");
        parentElement = document.getElementById("owner");
        parentElement.appendChild(dButton);
        buttonEle = document.getElementById("ytdl_btn");
        buttonEle.setAttribute("style", "border-radius: 30px;");
    }

    // Handle cases where the standard UI elements are not found
    if (!parentElement) {
        let msgCont =
            "
Oops, unable to attach button to the original location on the page - this seems to be some code/layout change by Youtube, for the time-being you can use the button below. Once this design is finalised and pushed to all by Youtube, an updated addon version will place the button to usual location.

You can read more about this here.

";
        showPopup("", msgCont, true);
        parentElement = document.getElementById("notificationPopup");
        parentElement.childNodes[2].appendChild(dButton);
    }

    // Additional UI handling omitted for brevity
}

This flexibility ensures that the extension remains functional even when YouTube updates its interface, providing a consistent user experience.

Written on December 3rd, 2024

Integrated Analysis of Video Stream Extraction and Downloading Mechanisms in Firefox Extensions

The provided JavaScript code pertains to two Firefox extensions, namely youtube_downloader-1.6.30 and easy_youtube_video_download-19.1. Both extensions are designed to facilitate the downloading of YouTube videos by extracting their stream URLs and saving them as local video files. This integrated analysis elucidates the methodologies and components employed by these extensions to accomplish their objectives, drawing insights from both versions of the code.

1. Event Handling and Messaging Infrastructure

Both extensions utilize an event-driven architecture to manage communication between different parts of the extension, such as content scripts and background scripts. This is achieved through dedicated modules and event listeners that handle asynchronous operations, ensuring seamless interaction and responsiveness within the extension's user interface.

a. Event Modules and Listeners

These modules are foundational in handling tasks such as fetching video information, responding to user interactions, and managing the download process.

var Events = (function () {
    chrome.runtime.onMessage.addListener(function (request, sender, sendResponse) {
        // Dispatching events based on message type
    });

    function _sendMessage(type, data, tab_id) {
        // Sending messages to specific tabs or the runtime
    }

    function _addListener(type, cb) {
        // Adding event listeners for specific message types
    }

    return {
        sendMessage: _sendMessage,
        addListener: _addListener,
        dispatchEvent: _dispatchEvent,
        addEventListener: _addEventListener
    };
})();

b. Initialization of Event Listeners

These listeners ensure that the extension responds appropriately to various events such as opening new tabs, retrieving video links, and handling signature decoders.

function main() {
    function openTab(ignore, data) {
        if (typeof data.url !== "undefined") {
            g_downloadManager.downloadVideo(data.url);
        }
    }

    Events.addListener("openBYDTab", openTab);

    Events.addListener("get_links", function (type, idVideo, ignore, tab_id) {
        g_downloadManager.getLinksForVideo(idVideo, tab_id);
    });

    Events.addEventListener("get_sig_decoder", function (event) {
        var data = event.data;
        if (data && data.href) {
            g_downloadManager.downloadSignatureDecoderAndDownloadLinks(data.href, function (data) {
                event.reply({
                    res: true,
                    data: data
                });
            }, function () {
                event.reply({
                    res: false
                });
            });
        }
    });
}

main();

2. User Interface Integration

The extensions seamlessly integrate with YouTube's interface by injecting download buttons and corresponding menus into the video watch page. This integration allows users to initiate the download process directly from the YouTube page, enhancing user experience and accessibility.

a. Injecting the Download Button

The download button is dynamically created and inserted into the YouTube page's DOM, adapting to different UI layouts to maintain functionality across various YouTube designs.

let dButton = document.createElement("button");
dButton.setAttribute("id", "ytdl_btn");
dButton.setAttribute("class", "ytdl_btn");
dButton.textContent = " Download: ▼ ";
dButton.setAttribute("data-tooltip-text", "Download Video");

// Check if the extension is on the old or new YouTube UI
let isOldUI = true;
if (document.getElementById("comment-teaser")) {
    isOldUI = false;
}

if (parentElement && isOldUI) {
    // OLD UI
    parentElement.childNodes[0].appendChild(dButton);
}

// Are we on a new design, generate and add new style button
if (parentElement && !isOldUI) {
    console.log("Inside new UI");
    parentElement = document.getElementById("owner");
    parentElement.appendChild(dButton);
    buttonEle = document.getElementById("ytdl_btn");
    buttonEle.setAttribute("style", "border-radius: 30px;");
}

if (!parentElement) {
    // Handle cases where the button could not be attached to the standard locations
    let msgCont =
        "
Oops, unable to attach button to the original location on the page - this seems to be some code/layout change by YouTube. For the time being, use the button below. Once the design is finalized and pushed to all by YouTube, an updated addon version will place the button in the usual location.

You can read more about this here.

";
    showPopup("", msgCont, true);
    parentElement = document.getElementById("notificationPopup");
    parentElement.childNodes[2].appendChild(dButton);
}

b. Creating the Download Menu

The download menu lists various available formats and qualities for the user to select, providing a user-friendly interface for initiating downloads.

// Create download list
let dList = document.createElement("div");
dList.setAttribute("id", "ytdl_list");
dList.setAttribute("status", "hide");
dList.classList.add("ytdl_list", "ytdl_list_hide");
dButton.appendChild(dList);

for (let i = 0; i < downloadCodeList.length; i++) {
    let getF = downloadCodeList[i].format;
    if (FORMAT_LABEL[getF]) {
        let linkDiv = document.createElement("div");
        linkDiv.setAttribute("class", "eytd_list_item");

        let dLink = document.createElement("a");
        let url = DOMPurify.sanitize(downloadCodeList[i].url);
        dLink.setAttribute("id", "ytdl_link_" + downloadCodeList[i].format);
        dLink.innerText = downloadCodeList[i].label;

        if (downloadCodeList[i].download || downloadCodeList[i].external) {
            dLink.setAttribute("href", url);
            if (downloadCodeList[i].label != "Settings") {
                dLink.setAttribute("download", downloadCodeList[i].download);
            }
            dLink.setAttribute("target", "_blank");
            if (!downloadCodeList[i].external) {
                dLink.addEventListener("click", notifyExtension, false);
            }
        } else {
            live("click", "ytdl_link_" + downloadCodeList[i].format, function () {
                var mp3_clean_url =
                    "https://videodroid.org/v3/authenticate.php?vid=" +
                    videoId +
                    "&stoken=" +
                    proKey +
                    "&format=" +
                    FORMAT_LABEL[getF] +
                    "&title=" +
                    videoTitle +
                    "&ver=" +
                    version;
                mp3_clean_url = encodeURI(mp3_clean_url);
                addiframe(mp3_clean_url, "250");
                return false;
            });
        }

        linkDiv.appendChild(dLink);
        dList.appendChild(linkDiv);
    }
}

var downloadBtn = document.getElementById("ytdl_btn");
downloadBtn.addEventListener("click", expandList);

3. Fetching and Parsing Video Information

Extracting downloadable video streams involves retrieving and parsing detailed video information from YouTube. Both extensions interact with YouTube's internal APIs and handle various data structures to obtain the necessary stream URLs and formats.

a. Fetching Video Data

This function constructs and sends a POST request to YouTube's internal API to retrieve video details, accommodating different client configurations and handling age-restricted content.

async function getInnerApijson(videoId, clientName, isAgeRestricted) {
    const clients = {
        "IOS_CREATOR": {
            clientDetails: {
                clientName: "IOS_CREATOR",
                clientVersion: "22.33.101",
                deviceModel: "iPhone14,3",
                userAgent: "com.google.ios.ytcreator/22.33.101 (iPhone14,3; U; CPU iOS 15_6 like Mac OS X)",
                hl: "en",
                timeZone: "UTC",
                utcOffsetMinutes: 0
            },
            apiKey: "AIzaSyA8eiZmM1FaDVjRy-dfKTyQ_vz_yYM39w"  // API key for IOS_CREATOR
        },
        "WEB": {
            clientDetails: {
                clientName: "WEB",
                clientVersion: "2.20201021.00.00",
                deviceModel: "",
                userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36",
                hl: "en",
                timeZone: "UTC",
                utcOffsetMinutes: 0
            },
            apiKey: "AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8"  // API key for WEB
        },
        "IOS": {
            clientDetails: {
                "clientName": "IOS",
                "clientVersion": "19.09.3",
                "deviceModel": "iPhone14,3",
                "userAgent": "com.google.ios.youtube/19.09.3 (iPhone14,3; U; CPU iOS 15_6 like Mac OS X)",
                "hl": "en",
                "timeZone": "UTC",
                "utcOffsetMinutes": 0
            },
            apiKey: "AIzaSyB-63vPrdThhKuerbB2N_l7Kwwcxj6yUAc"  // API key for IOS
        }
    };

    // Select the appropriate client configuration based on the input
    const { clientDetails, apiKey } = clients[clientName] || clients["IOS_CREATOR"]; // Default to IOS_CREATOR if clientName is not found

    // Customize client information based on age restriction
    const clientInfo = { ...clientDetails };
    if (isAgeRestricted) {
        clientInfo.clientVersion += "_restricted"; // Example suffix for age-restricted content
        clientInfo.clientScreen = "EMBED"; // This detail may be adjusted based on needs
    }

    // Construct the request body
    const requestBody = {
        context: { client: clientInfo },
        videoId: videoId,
        playbackContext: {
            contentPlaybackContext: {
                html5Preference: "HTML5_PREF_WANTS"
            }
        },
        contentCheckOk: true,
        racyCheckOk: true
    };

    // Define the fetch request details with the updated body structure
    const requestOptions = {
        method: "post",
        headers: {
            Accept: "application/json, text/plain, */*",
            "Content-Type": "application/json",
        },
        body: JSON.stringify(requestBody)
    };

    // Construct the fetch URL using the client's API key
    const url = `https://youtubei.googleapis.com/youtubei/v1/player?key=${apiKey}`;

    try {
        // Execute the fetch request
        const response = await fetch(url, requestOptions);
        if (!response.ok) {
            throw new Error(`HTTP error! Status: ${response.status}`);
        }
        return await response.json();
    } catch (error) {
        console.error("Failed to fetch video page:", error);
        return null; // Optionally return an error object or handle error differently
    }
}

b. Parsing Video Details

This function orchestrates the extraction process by identifying the video ID, fetching video data, handling potential age restrictions, deciphering signatures, and preparing the download options for user interaction.

async function parseDetails(url) {
    if (window.location.href.indexOf("shorts/") > -1) {
        let msgCont = "YouTube Shorts video uses a different UI and to download these videos you can use the Download menu provided via the Browser toolbar button.";
        showPopup("", msgCont, true);
        videoId = window.location.href.split("shorts/")[1];
    } else {
        const query = parseQueryString(url.split("?")[1]);
        videoId = query["v"];
    };

    let videoPage = await getInnerApijson(videoId, "IOS_CREATOR", false);

    if (!videoPage.streamingData) {
        // Seems age-gated video, refetch data
        videoPage = await getInnerApijson(videoId, "IOS_CREATOR", true);
        console.log("Using Age-gated Android");
    }

    // If it still fails, try using page data
    if (!videoPage.streamingData.formats) {
        videoPage = await getRawPageData();
        videoPage = JSON.parse(videoPage);

        // Save decsig function for usage later
        var basejs = window.localStorage.getItem("basejs");
        console.log("Scraping page data, and getting base.js : " + basejs)
        var decsig = await fetch(basejs)
            .then((res) => res.text())
            .then((body) => parseDecsig(body));
        console.log("YouTube Code Changed API Failed");
    }

    // We still have a failure, display error
    if (!videoPage.streamingData.formats) {
        let msgCont = "Error!!";
        // This could be a live video check and inform
        if (
            videoPage.videoDetails.isLive ||
            videoPage.playabilityStatus.reason == "This live event has ended."
        ) {
            msgCont =
                "
This is either an ongoing or recently finished Live stream. It can take up to 12-72 hours to generate download links for such videos. Please try again later.
";
            showPopup("", msgCont, true);
        } else {
            msgCont =
                "
Unable to parse the download links from this page. Please try refreshing the page. If this issue persists with all videos, please report it.
";
            showPopup("", msgCont, true);
        }
    }

    let videoTitle = document.title
        .replace(/^\(\d+\)\s*/, "")
        .replace(/\s*\-\s*YouTube$|'/g, "")
        .replace(/[\\/:"*?<>|#&]/g, "")
        .trim();

    const formatURLs = videoPage.streamingData.formats.map((format) => {
        let url = format.url;
        const cipher = format.signatureCipher || format.cipher;

        if (cipher) {
            const components = parseQueryString(cipher);
            const sig = decsig(components.s);
            url = `${components.url}&${encodeURIComponent(components.sp)}=${encodeURIComponent(sig)}`;
        }

        return {
            itag: format.itag,
            _decryptedURL: url,
        };
    });

    // Populate Adaptive Formats
    let adaptiveFormats;
    if (videoPage.streamingData.adaptiveFormats) {
        adaptiveFormats = videoPage.streamingData.adaptiveFormats;
    } else {
        // Fetch using another Client if adaptiveFormats are not available
        videoPage = await getInnerApijson(videoId, "IOS", false);
        adaptiveFormats = videoPage.streamingData.adaptiveFormats ? videoPage.streamingData.adaptiveFormats : [];
    }
    const adaptiveFormatURLs = videoPage.streamingData.adaptiveFormats.map(
        (format) => {
            let url = format.url;
            const cipher = format.signatureCipher || format.cipher;

            if (cipher) {
                const components = parseQueryString(cipher);
                const sig = decsig(components.s);
                url = `${components.url}&${encodeURIComponent(components.sp)}=${encodeURIComponent(sig)}`;
            }

            return {
                itag: format.itag,
                _decryptedURL: url,
            };
        }
    );

    const finalFmt = [...formatURLs, ...adaptiveFormatURLs];
    var downloadCodeList = await displayFMT(finalFmt, videoTitle);

    // Store download list and prepare UI elements
    sessionStorage.setItem("dList_" + videoId, JSON.stringify({ VideoData: downloadCodeList, videoTitle: videoTitle, key: proKey }));

    // Prepare button text based on language settings
    var language = document.documentElement.getAttribute("lang");
    language = language.substring(0, 2);
    var buttonText = BUTTON_TEXT[language]
        ? BUTTON_TEXT[language]
        : BUTTON_TEXT["en"];
    var buttonLabel = BUTTON_TOOLTIP[language]
        ? BUTTON_TOOLTIP[language]
        : BUTTON_TOOLTIP["en"];

    // Inject the download button into the YouTube page
    let dButton = document.createElement("button");
    dButton.setAttribute("id", "ytdl_btn");
    dButton.setAttribute("class", "ytdl_btn");
    dButton.textContent = " " + buttonText + ": ▼ " + " ";
    dButton.setAttribute("data-tooltip-text", buttonLabel);

    // Additional UI handling omitted for brevity
}

4. Signature Deciphering Mechanism

YouTube employs signature-based protection to secure its video streams. Both extensions incorporate mechanisms to parse and decipher these signatures, enabling the construction of valid download URLs.

a. Deciphering Function Extraction

This function analyzes YouTube's obfuscated JavaScript to extract and reconstruct the signature deciphering logic. By identifying and rebuilding the necessary functions, the extension can decode the signatures appended to video stream URLs, ensuring the validity and accessibility of the download links.

const parseDecsig = (data) => {
    try {
        const fnnameresult = /=([a-zA-Z0-9\$]+?)\(decodeURIComponent/.exec(data);
        const fnname = fnnameresult[1];
        const _argnamefnbodyresult = new RegExp(
            escapeRegExp(fnname) + "=function\\((.+?)\\){((.+)=\\2.+?)}"
        ).exec(data);
        const [_, argname, fnbody] = _argnamefnbodyresult;
        const helpernameresult = /;(.+?)\..+?\(/.exec(fnbody);
        const helpername = helpernameresult[1];
        const helperresult = new RegExp(
            "var " + escapeRegExp(helpername) + "={[\\s\\S]+?};"
        ).exec(data);
        const helper = helperresult[0];
        console.log(`parsedecsig result: ${argname}=>{${helper}\n${fnbody}}`);
        return new Function([argname], helper + "\n" + fnbody);
    } catch (e) {
        console.error("parsedecsig error:", e);
        console.info("script content:", data);
        console.info(
            'If you encounter this error, please copy the full "script content" to https://pastebin.com/ for me.'
        );
    }
};

b. Applying the Deciphered Signature

This mapping ensures that each video stream URL is properly decrypted and sanitized, making them ready for user-initiated downloads.

const formatURLs = videoPage.streamingData.formats.map((format) => {
    let url = format.url;
    const cipher = format.signatureCipher || format.cipher;

    if (cipher) {
        const components = parseQueryString(cipher);
        const sig = decsig(components.s);
        url = `${components.url}&${encodeURIComponent(components.sp)}=${encodeURIComponent(sig)}`;
    }

    return {
        itag: format.itag,
        _decryptedURL: url,
    };
});

5. Extracting and Organizing Download Links

After fetching and deciphering the necessary video information and signatures, the extensions process this data to generate a list of downloadable video streams in various formats and qualities.

a. Processing Video Formats

The displayFMT function processes the final list of video formats, sanitizes the URLs using DOMPurify, and organizes them into a structured list of download options. This list includes various video qualities and formats, as well as additional functionalities such as settings, contact, and donation links.

async function displayFMT(finalFmt, videoTitle) {
    let jsonData = {};
    let downloadCodeList = [];

    finalFmt.forEach((format) => {
        jsonData[format.itag] = format._decryptedURL;

        let fmt_url = format._decryptedURL;

        if (fmt_url && FORMAT_LABEL[format.itag]) {
            downloadCodeList.push({
                url: DOMPurify.sanitize(fmt_url),
                format: format.itag,
                label: FORMAT_LABEL[format.itag],
                download: `${videoTitle}.${FORMAT_TYPE[format.itag]}`,
            });
        }

        // Check availability of higher resolutions
        if (format.itag == "22") is720p = true;
        if (["136", "247"].includes(format.itag)) is720pDash = true;
        if (["137", "299", "303", "335", "617", "636"].includes(format.itag)) is1080p = true;
    });

    // Add additional high-quality download options
    if (is720pDash && !is720p) {
        downloadCodeList.push({
            url: DOMPurify.sanitize("https://videodroid.org/"),
            format: "720P",
            label: "MP4 720p (HD)",
        });
    }

    if (is1080p) {
        downloadCodeList.push({
            url: DOMPurify.sanitize("https://videodroid.org/"),
            format: "1080p3",
            label: "Full-HD 1080p",
        });
        is1080p = false;
    }

    // Add audio formats and additional options
    downloadCodeList.push(
        { url: DOMPurify.sanitize("https://videodroid.org/"), format: "mp3256", label: "MP3 HQ (256 Kbps)" },
        { url: DOMPurify.sanitize("https://videodroid.org/"), format: "mp3128", label: "MP3 HQ (128 Kbps)" },
        { url: browser.runtime.getURL("options/options.html"), format: "Settings", label: "Settings", external: true },
        { url: DOMPurify.sanitize("https://www.yourvideofile.org/support.html?&ver=" + version), format: "About", label: "Contact/Bug Report", external: true },
        { url: DOMPurify.sanitize("https://videodroid.org/pro_upgrade.html?&ver=" + version), format: "Donate", label: "Donate", external: true }
    );

    return downloadCodeList;
}

b. Generating Download Links

This loop iterates through the list of available formats, creating and appending download links to the download menu. It distinguishes between direct download links and external functionalities, ensuring that each option behaves as intended.

for (let i = 0; i < downloadCodeList.length; i++) {
    let getF = downloadCodeList[i].format;
    if (FORMAT_LABEL[getF]) {
        let linkDiv = document.createElement("div");
        linkDiv.setAttribute("class", "eytd_list_item");

        let dLink = document.createElement("a");
        let url = DOMPurify.sanitize(downloadCodeList[i].url);
        dLink.setAttribute("id", "ytdl_link_" + downloadCodeList[i].format);
        dLink.innerText = downloadCodeList[i].label;

        if (downloadCodeList[i].download || downloadCodeList[i].external) {
            dLink.setAttribute("href", url);
            if (downloadCodeList[i].label != "Settings") {
                dLink.setAttribute("download", downloadCodeList[i].download);
            }
            dLink.setAttribute("target", "_blank");
            if (!downloadCodeList[i].external) {
                dLink.addEventListener("click", notifyExtension, false);
            }
        } else {
            live("click", "ytdl_link_" + downloadCodeList[i].format, function () {
                var mp3_clean_url =
                    "https://videodroid.org/v3/authenticate.php?vid=" +
                    videoId +
                    "&stoken=" +
                    proKey +
                    "&format=" +
                    FORMAT_LABEL[getF] +
                    "&title=" +
                    videoTitle +
                    "&ver=" +
                    version;
                mp3_clean_url = encodeURI(mp3_clean_url);
                addiframe(mp3_clean_url, "250");
                return false;
            });
        }

        linkDiv.appendChild(dLink);
        dList.appendChild(linkDiv);
    }
}

5. Initiating Downloads

Upon user selection of a desired video format, the extensions initiate the download process using Chrome's downloads API. This allows users to save the video file locally in their chosen format and quality.

a. Handling Download Requests

This listener ensures that download requests are handled efficiently, providing feedback on the download status and alerting the user in case of errors such as illegal characters in filenames.

chrome.runtime.onMessage.addListener(function (message) {
    let fname = message.filename
        .trim()
        .replace(/[~!@#$%^&*()_|+\-=?;:'",<>{}[\]\\/]/gi, "-")
        .replace(/[\\/:*?"<>|]/g, "_")
        .substring(0, 240)
        .replace(/\s+/g, " ");

    chrome.downloads.download({
        url: message.url,
        filename: fname,
        conflictAction: "uniquify"
    }, function (downloadId) {
        if (typeof downloadId !== 'undefined') {
            console.log('Download initiated, ID is: ' + downloadId);
        } else {
            console.error('Download Error: ' + chrome.runtime.lastError.message);
            alert(chrome.runtime.lastError.message + ", This could be due to illegal characters in video title, try right-click and 'Save Link As' to download.");
        }
    });
});

b. Download Management Function

This function manages the download process by validating URLs, determining appropriate filenames, and utilizing Chrome's downloads API to initiate downloads while preventing duplicate requests within a short timeframe.

var _downloadVideo = (function () {
    var _lastDownloadedUrls = [];

    return function (url) {
        if (_lastDownloadedUrls.indexOf(url) < 0) {
            _lastDownloadedUrls.push(url);

            setTimeout(function () {
                // Prevent duplicate downloads
                var indexEl = _lastDownloadedUrls.indexOf(url);
                if (indexEl >= 0) {
                    _lastDownloadedUrls.splice(indexEl, 1);
                }
            }, 5000);

            setTimeout(function () {
                var params = {
                    "url": url,
                    "saveAs": true,
                    "method": "GET",
                    "conflictAction": "uniquify"
                };
                var filename = _getVideoNameFromUrl(url);
                if (filename) {
                    params["filename"] = filename;
                }
                chrome.downloads.download(params, function() {
                    console.error(chrome.runtime.lastError);
                });
            }, 300);
        }
    };
})();

6. Security and Sanitization

To maintain security and prevent potential vulnerabilities, the extensions employ sanitization mechanisms to cleanse URLs and user inputs. This is crucial in mitigating risks such as cross-site scripting (XSS).

a. Sanitizing URLs

The use of DOMPurify ensures that all URLs embedded into the DOM are sanitized, preventing the injection of malicious scripts or unwanted content.

let url = DOMPurify.sanitize(downloadCodeList[i].url);
dLink.setAttribute("href", url);

b. Validating User Inputs

These validations ensure that user-provided data, such as email addresses, adhere to expected formats, thereby enhancing the integrity and security of the extension.

function isValidEmail(email) {
    const re = /^(([^<>()\[\]\\.,;:\s@"]+(\.[^<>()\[\]\\.,;:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/;
    return re.test(String(email).toLowerCase());
}

emailInput.addEventListener("blur", (e) => {
    if (!isValidEmail(e.target.value) || e.target.value.trim() === "") {
        e.target.classList.add("is-invalid");
        errorMessage.innerText = "Invalid email address";
    } else {
        e.target.classList.remove("is-invalid");
        errorMessage.innerText = "";
    }
});

7. Additional Features and Enhancements

Beyond the core functionalities of extracting and downloading video streams, both extensions incorporate additional features to enhance user experience and provide extended functionalities.

a. File Size Calculation

This function calculates and displays the size of each downloadable file, providing users with information about the download before initiating it.

function addFileSize(url, format) {
    function updateVideoLabel(size, format) {
        var elem = document.getElementById("ytdl_link_" + format);
        if (elem) {
            size = parseInt(size, 10);
            if (size >= 1073741824) {
                size = parseFloat((size / 1073741824).toFixed(1)) + " GB";
            } else if (size >= 1048576) {
                size = parseFloat((size / 1048576).toFixed(1)) + " MB";
            } else {
                size = parseFloat((size / 1024).toFixed(1)) + " KB";
            }
            if (elem.childNodes.length > 1) {
                elem.lastChild.nodeValue = " (" + size + ")";
            } else if (elem.childNodes.length == 1) {
                elem.appendChild(document.createTextNode(" (" + size + ")"));
            }
        }
    }
    let matchSize = findMatch(url, /[&\?]clen=([0-9]+)&/i);
    if (matchSize) {
        updateVideoLabel(matchSize, format);
    } else {
        if (url.indexOf("googlevideo.com") !== -1) {
            fetch(url, {
                method: "HEAD",
            })
                .then((response) => {
                    let size = response.headers.get("content-length");
                    if (size) {
                        updateVideoLabel(size, format);
                    }
                })
                .catch((error) => {
                    console.log("Error Fetch Filesize URL : " + error);
                });
        }
    }
}

b. Dynamic UI Adjustments

These snippets ensure that the extension adapts to changes in YouTube's page structure, maintaining functionality even when YouTube updates its interface.

function insertAfter(el, referenceNode) {
    referenceNode.parentNode.insertBefore(el, referenceNode.nextSibling);
}

addEventListener("yt-page-data-updated", (event) => {
    restoreOptions();
    parseDetails(window.location.href);
    removeOldElements();
    let frm_div = document.getElementById("EXT_DIV");
    if (frm_div) {
        frm_div.remove();
    }
});

8. Integration and Initialization

Both extensions initialize their components and set up necessary event listeners to ensure seamless operation from the moment they are installed or activated.

a. Initialization of Options and Event Listeners

function restoreOptions() {
    chrome.storage.sync.get({
        autop: false,
        pKey: "",
        noNotify: false,
    }, function (items) {
        document.getElementById('autoplay').checked = items.autop;
        document.getElementById('prokey').value = items.pKey;
        document.getElementById('notification').checked = items.noNotify;
    });

    bootstart();
}

document.addEventListener('DOMContentLoaded', restoreOptions);
document.querySelector("form").addEventListener("submit", save_options);

These functions ensure that user preferences are loaded and applied upon the extension's activation, providing a personalized and consistent user experience.

9. Security and Compliance

Both extensions adhere to security best practices to protect user data and maintain compliance with browser policies.

a. Trusted Types and DOMPurify Integration

The integration of DOMPurify ensures that all user inputs and dynamically generated content are sanitized, preventing the injection of malicious scripts and safeguarding against XSS attacks.

/*! @license DOMPurify 2.3.1 | (c) Cure53 and other contributors | Released under the Apache license 2.0 and Mozilla Public License 2.0 | github.com/cure53/DOMPurify/blob/2.3.1/LICENSE */
// DOMPurify library code included for sanitizing HTML and URLs

b. Validating and Sanitizing Inputs

function isValidEmail(email) {
    const re = /^(([^<>()\[\]\\.,;:\s@"]+(\.[^<>()\[\]\\.,;:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/;
    return re.test(String(email).toLowerCase());
}

emailInput.addEventListener("blur", (e) => {
    if (!isValidEmail(e.target.value) || e.target.value.trim() === "") {
        e.target.classList.add("is-invalid");
        errorMessage.innerText = "Invalid email address";
    } else {
        e.target.classList.remove("is-invalid");
        errorMessage.innerText = "";
    }
});

By validating and sanitizing inputs, the extensions uphold high security standards, protecting both the user and the system from malicious exploits.

Written on December 3rd, 2024

nGeneFirefoxDownloader Extension Documentation v1.0 (Written December 3rd, 2024)

This document provides a comprehensive overview of the nGeneFirefoxDownloader Firefox extension, designed to facilitate the downloading of YouTube videos. Each component of the extension is presented with its corresponding source code followed by a detailed explanation to aid in understanding and further development.

1. manifest.json

{
  "name": "nGeneFirefoxDownloader",
  "description": "Simple YouTube video downloader for Firefox.",
  "manifest_version": 2,
  "version": "1.0",
  "icons": {
    "48": "icons/icon.png"
  },
  "background": {
    "scripts": ["background.js"]
  },
  "content_scripts": [
    {
      "js": ["content_script.js"],
      "matches": [
        "http://www.youtube.com/*",
        "https://www.youtube.com/*",
        "http://m.youtube.com/*"
      ],
      "exclude_matches": [
        "http://www.youtube.com/embed/*",
        "https://www.youtube.com/embed/*"
      ],
      "run_at": "document_end"
    }
  ],
  "permissions": ["downloads", "activeTab"],
  "browser_action": {
    "default_title": "nGeneFirefoxDownloader",
    "default_icon": {
      "16": "icons/icon.png",
      "32": "icons/icon32.png"
    }
  }
}

The manifest.json file serves as the blueprint for the Firefox extension, outlining its fundamental properties and configurations. Below is a breakdown of its key components:

• name: Specifies the name of the extension, which is "nGeneFirefoxDownloader" in this case.
• description: Provides a brief description of the extension's functionality, indicating that it is a simple YouTube video downloader for Firefox.
• manifest_version: Denotes the version of the manifest specification being used. Version 2 is utilized here, which is compatible with Firefox's extension framework.
• version: Indicates the current version of the extension, set to "1.0".
• icons: Defines the icon assets for the extension. A 48x48 pixel icon is specified, located at "icons/icon.png".
• background: Specifies background scripts that run in the background context. Here, "background.js" is listed as the background script.
• content_scripts: Details scripts that are injected into web pages matching specific patterns. The content_script.js is injected into YouTube pages (www.youtube.com and m.youtube.com) but excludes embed URLs (www.youtube.com/embed/*). The script executes at the end of the document loading process (document_end).
• permissions: Requests necessary permissions for the extension to function. The "downloads" permission allows the extension to initiate downloads, and "activeTab" grants access to the currently active tab.
• browser_action: Configures the extension's browser action, which includes:
  • default_title: Sets the tooltip text to "nGeneFirefoxDownloader".
  • default_icon: Specifies icons of different sizes (16x16 and 32x32 pixels) for the browser toolbar, enhancing visual recognition.

This manifest ensures that the extension is appropriately registered with Firefox, defines its operational scope, and requests the necessary permissions to perform video downloads from YouTube.

2. content_script.js

(function () {
  function getStreamingData() {
    // Try to get ytInitialPlayerResponse from the page
    let playerResponse = null;

    if (window.ytInitialPlayerResponse) {
      playerResponse = window.ytInitialPlayerResponse;
    } else {
      // Try to find it in the scripts
      let scripts = document.getElementsByTagName('script');
      for (let i = 0; i < scripts.length; i++) {
        let scriptContent = scripts[i].textContent;
        if (scriptContent.includes('ytInitialPlayerResponse')) {
          let jsonStr = scriptContent.match(
            /ytInitialPlayerResponse\s*=\s*(\{.*?\});/s
          );
          if (jsonStr && jsonStr[1]) {
            playerResponse = JSON.parse(jsonStr[1]);
            break;
          }
        }
      }
    }

    return playerResponse;
  }

  function getVideoTitle() {
    let titleElement = document.querySelector('h1.title');
    let title = titleElement ? titleElement.textContent.trim() : document.title.replace(' - YouTube', '').trim();
    return title;
  }

  function sanitizeFilename(name) {
    // Remove invalid characters
    name = name.replace(/[<>:"\/\\|?*\x00-\x1F]/g, '').trim();
    // Replace spaces with underscores
    name = name.replace(/\s+/g, '_');
    // Limit the length of the filename
    let maxFilenameLength = 100; // Adjust as needed
    if (name.length > maxFilenameLength) {
      name = name.substring(0, maxFilenameLength);
    }
    return name;
  }

  function addButton() {
    // Create the download button
    let downloadBtn = document.createElement('button');
    downloadBtn.textContent = 'Download Video';
    downloadBtn.style.position = 'fixed';
    downloadBtn.style.top = '10px';
    downloadBtn.style.right = '10px';
    downloadBtn.style.zIndex = 9999;
    downloadBtn.style.padding = '10px';
    downloadBtn.style.backgroundColor = '#ff0000';
    downloadBtn.style.color = '#ffffff';
    downloadBtn.style.border = 'none';
    downloadBtn.style.borderRadius = '5px';
    downloadBtn.style.cursor = 'pointer';
    document.body.appendChild(downloadBtn);

    downloadBtn.addEventListener('click', function () {
      let playerResponse = getStreamingData();
      if (!playerResponse) {
        alert('Failed to retrieve video data.');
        return;
      }

      let formats = playerResponse.streamingData.formats || [];
      let adaptiveFormats = playerResponse.streamingData.adaptiveFormats || [];

      // Merge formats
      let allFormats = formats.concat(adaptiveFormats);

      // Find 480p (itag 135) or 360p (itag 18 or 134)
      let targetFormat = allFormats.find(
        (f) => f.itag == 135 || f.itag == 18 || f.itag == 134
      );

      if (!targetFormat) {
        alert('480p or 360p format not available.');
        return;
      }

      let videoUrl = targetFormat.url;
      if (!videoUrl) {
        // Need to decipher signature
        let cipher = targetFormat.signatureCipher || targetFormat.cipher;
        if (cipher) {
          let urlParams = new URLSearchParams(cipher);
          let url = urlParams.get('url');
          let s = urlParams.get('s');
          let sp = urlParams.get('sp') || 'signature';
          if (s && url) {
            // Cannot decipher signature, so cannot download
            alert(
              'Cannot download this video due to signature encryption.'
            );
            return;
          } else {
            videoUrl = url;
          }
        } else {
          alert('Video URL not found.');
          return;
        }
      }

      let title = getVideoTitle();
      let filename = sanitizeFilename(title) + '.mp4';


      // Send message to background script to download the video
      chrome.runtime.sendMessage({
        action: 'downloadVideo',
        url: videoUrl,
        filename: filename
      });
    });
  }

  function init() {
    // Wait for the page to load
    if (document.readyState !== 'complete') {
      window.addEventListener('load', addButton);
    } else {
      addButton();
    }
  }

  init();
})();

The content_script.js file is responsible for interacting directly with YouTube web pages to facilitate the video downloading process. The script performs several key functions as outlined below:

1. Encapsulation:
   • The entire script is encapsulated within an Immediately Invoked Function Expression (IIFE) to avoid polluting the global namespace and ensure that variables and functions do not interfere with other scripts on the page.
2. Retrieving Streaming Data (getStreamingData):
   • Purpose: Extracts the streaming data necessary to identify available video formats and their respective URLs.
   • Methodology:
     1. Attempts to access ytInitialPlayerResponse directly from the window object.
     2. If unavailable, iterates through all <script> tags on the page to locate and parse the JSON object containing ytInitialPlayerResponse.
     3. Returns the parsed playerResponse object containing streaming data or null if unsuccessful.
3. Fetching Video Title (getVideoTitle):
   • Purpose: Obtains the title of the currently viewed YouTube video to use as the filename for the downloaded video.
   • Methodology:
     1. Attempts to select the <h1> element with the class title.
     2. If the element is found, extracts and trims its text content.
     3. If not, defaults to using the document's title, removing the " - YouTube" suffix.
4. Sanitizing Filename (sanitizeFilename):
   • Purpose: Cleans the video title to create a valid filename by removing prohibited characters and formatting the string.
   • Methodology:
     1. Removes characters that are invalid in filenames (<>:"/\|?* and control characters).
     2. Replaces spaces with underscores to enhance readability and compatibility.
     3. Truncates the filename to a maximum length (e.g., 100 characters) to prevent filesystem errors.
5. Adding the Download Button (addButton):
   • Purpose: Injects a "Download Video" button into the YouTube page interface, enabling users to initiate the download process.
   • Methodology:
     1. Creates a <button> element with the text "Download Video".
     2. Styles the button with fixed positioning, prominent colors, padding, and other CSS properties to ensure visibility and usability.
     3. Appends the button to the document's body.
     4. Attaches a click event listener to handle the download process when the button is pressed.
6. Download Process on Button Click:
   • Steps:
     1. Retrieve Streaming Data: Calls getStreamingData() to obtain the necessary video data.
     2. Error Handling: Alerts the user if streaming data cannot be retrieved.
     3. Format Selection: Merges available video formats and selects the desired quality (e.g., 480p or 360p) based on specific itag values.
     4. URL Extraction:
        • Attempts to extract the direct video URL from the selected format.
        • If the URL is not directly available, handles signature encryption scenarios by parsing the signatureCipher or cipher fields.
        • Alerts the user if the video URL cannot be determined due to encryption.
     5. Filename Preparation: Retrieves and sanitizes the video title to create a suitable filename with an .mp4 extension.
     6. Message Dispatch: Sends a message to the background script with the action downloadVideo, including the video URL and the sanitized filename.
7. Initialization (init):
   • Purpose: Ensures that the download button is added only after the page has fully loaded.
   • Methodology:
     1. Checks the document's readiness state.
     2. If the document is not yet complete, adds an event listener for the load event to call addButton().
     3. If the document is already loaded, calls addButton() immediately.

This content script effectively integrates a user interface element into YouTube pages, enabling users to download videos by interacting with the injected button. It handles the extraction and sanitization of necessary data while ensuring compatibility and error handling for various scenarios.

3. background.js

chrome.runtime.onMessage.addListener(function (request, sender, sendResponse) {
  if (request.action === 'downloadVideo') {
    chrome.downloads.download(
      {
        url: request.url,
        filename: 'Videos/' + request.filename, // This will save to a 'Videos' folder within the default download directory
        saveAs: false
      },
      function (downloadId) {
        if (downloadId) {
          console.log('Download initiated with ID:', downloadId);
        } else {
          console.error('Download failed:', chrome.runtime.lastError);
        }
      }
    );
  }
});

The background.js file operates within the extension's background context, handling tasks that require persistent operation, such as initiating downloads. Below is an elucidation of its functionality:

1. Message Listener (chrome.runtime.onMessage.addListener):
   • Purpose: Listens for messages dispatched from other parts of the extension, specifically from the content script.
   • Functionality:
     • Receives messages containing various actions and associated data.
     • Checks if the received message's action property matches 'downloadVideo', indicating a request to download a video.
2. Handling downloadVideo Action:
   • Parameters:
     • url: The direct URL to the video file to be downloaded.
     • filename: The sanitized name for the downloaded video file.
   • Operation:
     • Utilizes the chrome.downloads.download API to initiate the download process.
     • Download Options:
       • url: Specifies the direct link to the video to be downloaded.
       • filename: Determines the path and name of the downloaded file. In this case, the video is saved within a "Videos" folder in the user's default download directory, with the filename provided by the content script.
       • saveAs: Set to false to bypass the "Save As" dialog, allowing the download to proceed automatically.
   • Callback Function:
     • Success: If the download is successfully initiated, logs the download ID to the console for reference.
     • Failure: If the download fails to start, logs an error message detailing the reason, utilizing chrome.runtime.lastError for diagnostic purposes.

This background script plays a crucial role in managing the download operations triggered by the user through the content script. By handling messages and interfacing with the Chrome Downloads API, it ensures that video files are downloaded efficiently and appropriately, adhering to the specified parameters.

Written on December 3rd, 2024

Blockchain

ETH Payment for Software Sales (Written November 26, 2025)

Enabling real Ethereum payments for software sales via a web browser involves coordinating blockchain transactions with a user-friendly interface and secure backend processes. This comprehensive guide outlines the essential components and best practices – from integrating a wallet for user payments, choosing the right Ethereum network for testing and production, setting up payment triggers, delivering software securely after payment (e.g., via email), to designing both simple and full-fledged DApp implementations. Proper accounting and security (including transaction signing and broadcasting) are emphasized throughout to ensure a reliable and trustworthy payment system.

Wallet Integration

To accept Ethereum payments through a browser, you need to connect your web application to an Ethereum wallet. There are two main approaches for wallet integration:

Browser Wallet (e.g., MetaMask)

A browser wallet is a wallet application (often a browser extension or built-in browser feature) that users install to manage their Ethereum accounts. MetaMask is a prime example, providing an Ethereum provider (window.ethereum) to web pages. When integrated, your application can prompt the user to connect their MetaMask wallet and request transactions. The user’s private keys never leave the wallet; they review and authorize the transaction via the wallet’s interface. This approach leverages well-tested software and offers a familiar experience to users already involved in the Ethereum ecosystem.

Pros: Using a browser wallet is convenient and secure since key management is handled by the wallet. The developer can use standard Web3 libraries (such as ethers.js) to interact with EIP-1193 compatible providers like MetaMask. It’s easier to implement because much of the complexity (account management, signing, network connectivity) is handled by the wallet. Users also tend to trust known wallets, and no sensitive keys are exposed to the web application.

Cons: It requires the user to have the wallet installed and set up. New users might find this as an extra onboarding step. You are also limited by the wallet’s availability and the user’s willingness to use it; if a user does not have MetaMask (or a compatible wallet), they must install it before proceeding. Additionally, the interface and transaction prompts are controlled by the wallet, so customization of the signing experience is limited.

Custom Wallet Integration

Custom wallet integration means building your own solution for handling Ethereum transactions within the application or integrating a non-browser wallet solution. This could involve using a library to generate and manage Ethereum keys in the browser or implementing WalletConnect to connect with mobile wallets. In a fully custom approach, the application itself might hold the logic to create and sign transactions (using user-provided keys or an embedded wallet).

Pros: A custom integration can streamline the user experience by not requiring an external extension – for example, the app could handle payments directly or use a mobile wallet scan. It offers flexibility to tailor the wallet interface or support special use cases. This might be useful in a controlled environment or for integrating hardware wallets or other identity solutions.

Cons: Managing keys and transactions on your own is complex and risky. The application must securely handle private keys (which is a significant security responsibility). Any vulnerability could compromise users’ funds. Development effort is much higher: you need to implement key storage (preferably encrypted, perhaps using the Web Crypto API or secure hardware modules), transaction signing, and network communication. Furthermore, convincing users to trust a custom wallet may be difficult compared to well-known wallets. For most cases, relying on established wallets (like MetaMask or others such as Coinbase Wallet, Brave Wallet, etc.) is recommended unless there’s a compelling reason to build a custom solution.

Network Deployment: Mainnet vs. Testnets

When building an Ethereum payment system, you must decide where to deploy and test your solution. Ethereum offers a mainnet (the main network where real-value transactions occur) and several testnets (public testing networks like Goerli or Sepolia that use worthless test Ether). Development should occur on testnets, while actual software sales transact on the mainnet.

• Ethereum Mainnet (Production): The Ethereum mainnet is the live network where Ether has real monetary value. Deploying your payment system here means real ETH will be used for purchases. The mainnet offers permanence and security – transactions are final and carry real economic weight. However, using mainnet comes with considerations: gas fees (transaction fees) can be high during network congestion, and mistakes are costly (funds lost due to bugs or accidental transactions are real losses). For production software sales, mainnet is necessary to receive actual revenue, but it should only be used once the system has been thoroughly tested and secured.
• Ethereum Testnets (Development): Testnets like Goerli and Sepolia are replicas of Ethereum’s environment but with test tokens that have no real value. They are invaluable for development and testing. You should deploy smart contracts and run payment flows on a testnet first to ensure everything works correctly. Pros: no risk of financial loss, free test ETH (from faucets) to simulate payments, and the ability to experiment freely. Cons: testnets might have lower reliability or different performance characteristics than mainnet, and they are periodically reset or deprecated (for example, developers migrated from older testnets like Rinkeby to Goerli/Sepolia). Despite minor differences, a successful test on a modern testnet is a strong indicator that the system will work on mainnet. Always use a testnet for iterative development, then switch to mainnet for the actual launch.

Note: Maintain separate configurations for testnet and mainnet (e.g., different contract addresses and RPC endpoints). Test thoroughly on a testnet before deploying to mainnet, as this can catch issues early and prevent costly mistakes with real ETH.

Payment Triggers

“Payment trigger” refers to how the user’s action in the browser initiates the Ethereum payment and how the transaction is verified. There are two models to consider:

Direct Button Trigger (Simple Transaction)

In this approach, the web interface includes a payment button (e.g., “Pay with Ethereum”). When clicked, it directly requests a transaction from the user’s wallet. For instance, using a browser wallet, your application might call ethereum.request({ method: 'eth_sendTransaction', ... }) with the recipient (your business’s Ethereum address) and the price amount. The user then confirms the transaction in their wallet, sending ETH directly to your address. The application can monitor the transaction status (via the wallet or by querying the blockchain) and, once it’s mined, proceed to record the payment and deliver the product.

• Pros: This method is straightforward – no custom smart contract is needed if you are simply receiving funds to a wallet. The implementation is mostly on the front-end and possibly a backend to confirm and log the payment. It has less overhead in terms of gas (just a standard ETH transfer). Fewer moving parts mean fewer potential points of failure in the blockchain logic.
• Cons: All purchase logic (like ensuring the correct amount or handling multiple product types) is off-chain in the application code, which means it’s not enforced by the blockchain. The user must trust that the website will deliver the software after receiving payment (there’s no automatic enforcement on-chain). Similarly, the seller might manually have to verify transactions or rely on back-end logic to confirm payment. Features like refunds or purchase history may need manual handling or additional coding since the blockchain transaction alone is just a value transfer with no product context.

Smart Contract-Based Validation

Here, payments are handled by a smart contract which contains the purchase logic. Instead of sending ETH to an externally owned address, the user calls a function on a deployed smart contract (for example, purchase() or buyLicense(productId)) and includes the payment in that transaction. The smart contract can be programmed to validate the payment (check that the attached ETH equals the price, ensure the purchase hasn’t been made before for that user, etc.) and then emit an event or update its state to record the sale. The web application listens for this confirmation (either by waiting for the transaction success or by watching for the contract’s event) and then triggers software delivery.

• Pros: Using a smart contract can increase trust and automation. The contract acts as an arbiter that only accepts the transaction if conditions are met (correct payment amount, valid product identifier, etc.). It provides a public record of the purchase on the blockchain – anyone (including the customer) can verify that the payment was received by the contract and, if coded, that it was attributed to that customer or product. Smart contracts can also handle additional logic, such as automatically forwarding funds to a treasury account, or even holding funds in escrow until certain conditions are met. They could facilitate features like refund functions or discount logic transparently. Overall, this approach can make the system more robust and self-contained in the blockchain realm (except for the actual file delivery).
• Cons: Writing and deploying a smart contract introduces complexity. You must develop the contract in Solidity (or another language), test it extensively (since bugs can be costly), and possibly have it audited for security if handling significant funds. Each transaction to a contract costs gas according to the operations performed, which will be slightly higher than a simple transfer – meaning customers will pay a bit more in fees. If requirements change (prices, features), updating a smart contract is not trivial; it may require deploying a new version. Additionally, while the contract can record a purchase, it still cannot deliver the software itself; you will need an off-chain process triggered by the contract’s events. Thus, the contract adds value in validation and record-keeping, but the overall system still needs off-chain integration for full functionality.

Note: Even with a smart contract handling payment validation, the user experience still begins with a button click in the browser. The difference lies in what that click does under the hood. Ensure your UI clearly indicates the progress (e.g., “Waiting for transaction confirmation...”) since contract interactions might take longer than a simple transfer and involve additional confirmation steps.

Software Delivery and Post-Payment Email Integration

Once a payment is successful, the software (or license key) needs to be delivered to the customer. A common method for digital products is delivery via email. Integrating email into the post-payment process requires bridging the on-chain event to an off-chain action in a secure way:

1. Collect Customer Email: The process should start by obtaining the buyer’s email, either through a form at checkout or via prior account registration. This email will be used to send the product. It’s important to tie this email to the payment session (e.g., save it along with an order ID or the buyer’s wallet address before initiating the payment).
2. Process Payment on Blockchain: The user initiates the Ethereum payment (whether via direct transfer or smart contract call, as discussed above) through the browser. The interface can display the payment details, and the user confirms the transaction with their wallet. The transaction is then broadcast to the blockchain.
3. Detect Payment Confirmation: Your system needs to know when the payment has been successfully received. This can be done in a couple of ways. The front-end can listen for the transaction receipt via the Web3 provider (so it knows when the transaction is mined). More robustly, a back-end service or cloud function can monitor the blockchain. For example, if using a smart contract, you can listen for a specific event (like PurchaseCompleted(address buyer, ...)). If using a direct payment, the back end could watch the address for an incoming transfer of the exact amount and from the expected user address.
4. Trigger Email Dispatch: After confirming the payment on-chain (and possibly waiting for a few block confirmations for security), the system triggers an email to the buyer. The email contains the software download link, license key, or other instructions to access the product. Use a secure email service or SMTP server to send this message. At this step, it’s crucial to ensure the content is delivered only to the paying customer. One approach is to generate a unique, hard-to-guess download URL for each purchase and include that in the email rather than a public link on the website.
5. Secure Delivery: The download link or attached software should be protected. If a link is provided, it can be a one-time use link or token that expires after first download or after a certain time frame. This prevents anyone who might obtain the URL from reusing it or sharing it. Additionally, the email itself should clarify any steps the user needs to take and provide support contact info if there are issues.

Security in this process hinges on the back-end integration. The server or cloud function that sends the email must verify the payment thoroughly. For example, it should check that the amount paid matches the expected price, that the payment went to the correct address or smart contract, and (if applicable) that the purchase hasn’t already been processed to avoid duplicate emails. By cross-referencing the transaction details with the saved order data (like the email and maybe an order ID), the system can confidently deliver the software to the right recipient.

Note: Smart contracts themselves cannot send emails or interact with external systems directly. Any on-chain payment event must be picked up by an off-chain component. This could be a simple server-side script polling for events or using webhooks from a blockchain API service. Ensuring this communication is reliable and secure (using authenticated calls to your email service, etc.) is part of the integration effort.

Development Scope: Minimal vs. Full-Featured Implementation

The complexity of your Ethereum payment integration can vary. It’s useful to distinguish between a minimal viable implementation and a full-featured decentralized application (DApp) approach:

Minimal Implementation – ETH Transfer & Logging

In the simplest case, you might not deploy any smart contracts at all. The workflow could be: a user clicks a “Pay with ETH” button on your website, the site requests the user’s wallet to send a fixed amount of ETH to your company’s Ethereum address, and that’s it on the blockchain side. On the back end, you log the transaction and then manually or semi-automatically deliver the software.

Even in this minimal scenario, good practice is to record the payment in your system’s database or ledger. Each transaction should generate a record including who paid (e.g., the Ethereum address and the email), how much was paid, and for what product. Importantly, you would use double-entry bookkeeping principles to log the financial aspect. For example, if a customer pays 1 ETH for a software license:

This journal entry indicates that your crypto wallet (an asset account) has increased by 1 ETH, and a corresponding credit is made to a revenue account for the sale. Proper logging like this ensures your financial records remain accurate and audit-ready. In a minimal implementation, such logging might be done manually or via a simple script after detecting the payment.

Advantages of the minimal approach include its low implementation effort – you mostly rely on existing wallet functionality and perhaps basic scripts or services to watch for transactions. There are fewer components to secure (no smart contract code to write or audit). However, the trade-off is that many processes may be manual or not trust-minimized. For instance, you might manually verify that a transaction appeared and then send the email with the product. This is acceptable for low volume or initial testing but can become cumbersome or error-prone at scale.

Full-Featured DApp – Smart Contract, Frontend, & Integrated Wallet

A full-featured solution transforms the process into a more automated and blockchain-integrated DApp. In this setup, you will develop a frontend application (web UI) that interacts with a smart contract deployed on Ethereum. Users still use a wallet (like MetaMask) to sign transactions, but the difference is the logic for purchasing is baked into the smart contract.

For example, you might create a Solidity contract named SoftwareStore with functions such as buySoftware(uint productId) which is payable. The contract might contain a price list or accept a payment amount and associate it with the product ID. When a user initiates a purchase on the website, the DApp will call this buySoftware function via the user’s wallet. The contract, upon receiving the payment, emits an event (say LicensePurchased(address buyer, uint productId)). The event and the transaction outcome serve as an on-chain receipt.

The front end can listen for the transaction confirmation, giving immediate feedback to the user (e.g., “Transaction confirmed, thank you for your purchase!”). Meanwhile, your back end (or a serverless function) also listens for the LicensePurchased event. When it detects it, it automatically triggers the email with the license or download link, similar to the earlier description. In a full DApp scenario, you could even enhance this by having the smart contract issue a token or NFT representing the license, or allow the user to later cryptographically prove ownership of a license by showing they made a transaction.

Such a system is highly automated: the blockchain transaction itself serves as both payment and record of sale, and your off-chain system just bridges the final delivery. It’s more trustless, because the user knows that if the transaction succeeded on the blockchain, the contract logic was executed (for example, you might program the contract to allow the user to download from a decentralized storage if applicable, though email is simpler for most software distribution). The transparency is greater – both parties can see the transaction details publicly. Additionally, future extensions are possible, like adding referral rewards, on-chain tracking of licenses, or integration with other dApps (perhaps a marketplace of software licenses).

The drawbacks of the full-featured approach are the higher development and maintenance effort. You must ensure the smart contract is secure and updated when needed. Users will have to pay gas fees that correspond to the contract’s operations, which could be slightly more than a direct send. There’s also a learning curve for users new to DApps – connecting wallets, etc., but as crypto adoption grows this is less of an issue. Overall, for a production system with significant volume, investing in a full DApp architecture can pay off in automation and reliability.

Secure Transaction Signing & Broadcasting

No matter which integration path is chosen, handling Ethereum transactions securely is essential. Below are key practices for signing and broadcasting payments safely through a browser-based system:

• Using Wallets for Signing: In a browser scenario, leveraging wallets like MetaMask is the safest route. When the user initiates a payment, MetaMask (or a similar wallet) will present the transaction details to the user for approval. The wallet uses the user’s private key (stored securely within it) to sign the transaction. The web application never directly accesses the private key, which is a critical security advantage. Always ensure the transaction details (recipient address, amount of ETH, and if applicable, contract data) are correct and clearly shown to the user in the wallet confirmation window. This helps prevent mistakes and phishing attempts.
• Custom Signing Considerations: If your implementation involves custom signing (say, you built your own wallet interface or you are signing transactions in a backend service), take extreme care with private key management. Use well-known libraries such as ethers.js or web3.js to construct and sign the transaction – these libraries handle nuances like proper transaction formatting and include the chain ID to prevent replay attacks. Never transmit or expose the raw private key. If signing in the browser (not recommended unless absolutely necessary), the key could be derived from a user-entered seed phrase or imported file and should only live in memory. If signing on the server (for instance, the server might need to sign a transaction to send a refund or to move received funds), store keys in a secure manner (environment variables or a vault, and encrypted at rest). Limit access to these keys and use access controls so that even if your database is compromised, the keys remain safe.
• Broadcasting the Transaction: After signing, the transaction must be sent to the Ethereum network. A browser wallet will usually handle this automatically by sending through its connected node. If doing it manually, you can use an RPC endpoint (like Infura, Alchemy, or your own Ethereum node) by sending the raw signed transaction via an API call (eth_sendRawTransaction). Ensure you connect to the correct network (testnet or mainnet as appropriate) before broadcasting. Monitor the response – you should receive a transaction hash if successfully accepted into the mempool. It’s good practice to show this transaction hash to the user and perhaps link to a block explorer so they can track it.
• Confirmation & Error Handling: Design your application to handle the asynchronous nature of blockchain transactions. After broadcasting, there is a period where the transaction is pending. The app should reflect this (e.g., “Payment pending, awaiting confirmation...”). Once mined, the app and/or backend should verify that the transaction succeeded (not reverted) and then proceed with delivery. Use the transaction receipt to double-check the outcome. In case of a failure (transaction rejected or out of gas), you should catch that and inform the user, possibly allowing them to retry. For important transactions, consider waiting for multiple confirmations (several blocks) to guard against the extremely rare chance of chain reorganization on Ethereum. One confirmation is typically sufficient for irreversibility on Ethereum, but a few confirmations can provide extra assurance for high-value sales.
• Security Best Practices: Always encourage users to practice wallet safety – for instance, they should double-check that the website they are using is the legitimate one (to avoid phishing sites that could trick them into sending funds to the wrong address). Your site should be served over HTTPS to prevent any man-in-the-middle tampering with the wallet connection or injected scripts. If feasible, integrate support for hardware wallets, since they require physical confirmation and are safer for the user’s private keys. Keep your dependencies (like Ethereum libraries) up to date to benefit from the latest security patches. Internally, log transaction activities and monitor for anomalies (e.g., multiple failed payment attempts) which might indicate issues.