process MathML nested inside mtext

[xworld21]

Is your feature request related to a problem? Please describe.
The <mtext> element is allowed to contain arbitrary HTML, and this is now properly supported via allowHtmlInTokenNodes. There are two inconsistencies, though:

1. if a <math> element appears as direct children of <mtext>, MathJax returns the error 'Improper nesting of math tags'
2. otherwise, <math> is accepted, but MathJax does not process it

Describe the solution you'd like
<math> descendants of <mtext> should be processed like any other <math>.

Describe alternatives you've considered
To work around 1, it is enough to generate better MathML and split <mtext>, or to wrap the content into a further <div>. For 2, MathML support in browsers is not too bad. So all in all this is a minor issue.

Additional context
This is a rare circumstance, as far as I know. The mathtools documentation has an (artificial!) LaTeX example that can't be replicated properly without nesting MathML-in-HTML-in-MathML, but I haven't encountered it in practice.

[dpvc]

Here is a configuration that will do as you suggest for (1) and break the mtext element in two with contents of the math element in between. Perhaps that will suffice for your situation.

MathJax = {
  mml: {
    mmlFilters: [
      ({data, document}) => {
        //
        //  Loop through any nested math elements...
        //
        for (const math of data.querySelectorAll('math')) {
          //
          //  Skip if they are in a nested HTML element
          //
          if (math.parentNode.namespaceURI !== 'http://www.w3.org/1998/Math/MathML') continue;
          //
          //  Break the token node into two at the math node
          //
          const first = math.parentNode;
          const last = first.cloneNode();
          while (first.lastChild !== math) {
            last.insertBefore(first.lastChild, last.firstChild);
          }
          first.removeChild(first.lastChild);
          //
          //  Make sure spaces at the break are retained
          //
          let text = first.lastChild;
          if (text?.nodeName === '#text') {
            text.textContent = text.textContent.replace(/\s+$/, '\u00A0');
          }
          text = last.firstChild;
          if (text?.nodeName === '#text') {
            text.textContent = text.textContent.replace(/^\s+/, '\u00A0');
          }
          //
          //  If there are attributes on the math node, copy them to an mstyle
          //  node, otherwise use an mrow to replace the math node
          //
          if (math.attributes.length) {
            const mstyle = document.adaptor.node('mstyle', {}, math.childNodes);
            for (const {name, value} of math.attributes) {
              mstyle.setAttribute(name, value);
            }
            first.after(mstyle);
            mstyle.after(last);
          } else {
            const mrow = document.adaptor.node('mrow', {}, math.childNodes);
            first.after(mrow);
            mrow.after(last);
          }
        }
      }
    ]
  },
  loader: {
    'input/mml': {
      ready() {
        //
        // Subclass the FindMathML class to remove any nested
        // math elements, since they are processed above
        //
        const {FindMathML} = MathJax._.input.mathml.FindMathML;
        class FindMML extends FindMathML {
          processMath(set) {
            for (const mml of set.values()) {
              const parent = mml.parentNode;
              if (parent.namespaceURI === 'http://www.w3.org/1998/Math/MathML') {
                set.delete(mml);
              }
            }
            return super.processMath(set);
          }
        }
        MathJax.config.mml.FindMathML = new FindMML();
      }
    }
  }
};

Note that yo don't need allowHtmlInTokenNodes: true for this, since math is a MathML node, and so will be handled without that.

This code assumes you are working in the browser; that is, it assumes a full DOM implementation. If you are working in a command-line node application, it would need to be modified to use a DOM adaptor instead.