The following MWE creates two slides with Text 1, Text 2 as long as notes are not enabled. Once notes are enabled (commented line below), compiling the document produces Text 1 on the first slide and an empty second slide. The small preview in the corner of slide 2's note page does show Text 2, but the slide itself is blank.
\documentclass{beamer}
\usepackage{pgfpages}
% \setbeameroption{show notes on second screen=right}
\begin{document}
\begin{frame}
\only<1>{Text 1}
\only<2>{Text 2}
\end{frame}
\end{document}
The problem seems specific to xelatex: things work fine with pdflatex and lualatex.
XeTeX 3.14159265-2.6-0.999991 (TeX Live 2019/Debian)
kpathsea version 6.3.1
Copyright 2019 SIL International, Jonathan Kew and Khaled Hosny.
There is NO warranty. Redistribution of this software is
covered by the terms of both the XeTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the XeTeX source.
Primary author of XeTeX: Jonathan Kew.
Compiled with ICU version 66.1; using 66.1
Compiled with zlib version 1.2.11; using 1.2.11
Compiled with FreeType2 version 2.10.1; using 2.10.1
Compiled with Graphite2 version 1.3.13; using 1.3.13
Compiled with HarfBuzz version 2.6.4; using 2.6.4
Compiled with libpng version 1.6.37; using 1.6.37
Compiled with poppler version 0.68.0
Compiled with fontconfig version 2.13.1; using 2.13.1
Anonymous