diff --git a/src/Plugins/SimplnxCore/docs/ComputeBoundingBoxStatsFilter.md b/src/Plugins/SimplnxCore/docs/ComputeBoundingBoxStatsFilter.md index 5901b84026..0499410ff8 100644 --- a/src/Plugins/SimplnxCore/docs/ComputeBoundingBoxStatsFilter.md +++ b/src/Plugins/SimplnxCore/docs/ComputeBoundingBoxStatsFilter.md @@ -6,7 +6,7 @@ Statistics ## Description -***NOTE:* This filter allows each **Cell** value to contribute to 0 or more bounding boxes (a 1-to-many architecture). This comes with the caveat that the *Standardize Data* option offered by [Compute Attribute Array Statistics](ComputeArrayStatisticsFilter.md) cannot be included as functionality in this filter.** +**NOTE:** This filter allows each **Cell** value to contribute to 0 or more bounding boxes (a 1-to-many architecture). This comes with the caveat that the *Standardize Data* option offered by [Compute Attribute Array Statistics](ComputeArrayStatisticsFilter.md) cannot be included as functionality in this filter. This filter computes a variety of statistics for a given scalar array, restricted to the **Cells** that fall inside each supplied bounding box. Because boxes may overlap (or leave gaps), a single **Cell** may be counted toward several boxes, exactly one box, or none. The currently available statistics are array length, minimum, maximum, (arithmetic) mean, median, mode, standard deviation, summation, and number of unique values; any combination of these statistics may be computed. Any scalar array, of any primitive type, may be used as input. The type of each output array depends on the kind of statistic computed: diff --git a/src/Plugins/SimplnxCore/docs/ComputeKMeansFilter.md b/src/Plugins/SimplnxCore/docs/ComputeKMeansFilter.md index 956a217c4f..47150cf811 100644 --- a/src/Plugins/SimplnxCore/docs/ComputeKMeansFilter.md +++ b/src/Plugins/SimplnxCore/docs/ComputeKMeansFilter.md @@ -6,7 +6,7 @@ DREAM3D Review (Clustering) ## Description -***Warning:* The randomness in this filter is not currently consistent between operating systems even if the same seed is used. Specifically between Unix and Windows. This does not affect the results, but the IDs will not correspond. For example if the Cluster Identifier at index one on Linux is 1 it could be 2 on Windows, the overarching clusters will be the same, but their IDs will be different.** +**Warning:** *The randomness in this filter is not currently consistent between operating systems even if the same seed is used. Specifically between Unix and Windows. This does not affect the results, but the IDs will not correspond. For example if the Cluster Identifier at index one on Linux is 1 it could be 2 on Windows, the overarching clusters will be the same, but their IDs will be different.* This **Filter** applies the k means algorithm to an **Attribute Array**. K means is a *clustering algorithm* that assigns each point (each tuple) of the **Attribute Array** to a *cluster Id*. A **cluster** is a group of data points that are more similar to each other than to points in other groups. The user must specify the number of clusters (a dimensionless count, *k*) in which to partition the array. diff --git a/src/Plugins/SimplnxCore/docs/ComputeKMedoidsFilter.md b/src/Plugins/SimplnxCore/docs/ComputeKMedoidsFilter.md index 427e3fa198..cc27fba6ea 100644 --- a/src/Plugins/SimplnxCore/docs/ComputeKMedoidsFilter.md +++ b/src/Plugins/SimplnxCore/docs/ComputeKMedoidsFilter.md @@ -6,7 +6,7 @@ DREAM3D Review (Clustering) ## Description -***Warning:* The randomness in this filter is not currently consistent between operating systems even if the same seed is used. Specifically between Unix and Windows. This does not affect the results, but the IDs will not correspond. For example if the Cluster Identifier at index one on Linux is 1 it could be 2 on Windows, the overarching clusters will be the same, but their IDs will be different.** +**Warning:** *The randomness in this filter is not currently consistent between operating systems even if the same seed is used. Specifically between Unix and Windows. This does not affect the results, but the IDs will not correspond. For example if the Cluster Identifier at index one on Linux is 1 it could be 2 on Windows, the overarching clusters will be the same, but their IDs will be different.* This **Filter** applies the k medoids algorithm to an **Attribute Array**. K medoids is a *clustering algorithm* that assigns each point (each tuple) of the **Attribute Array** to a *cluster Id*. A **cluster** is a group of data points that are more similar to each other than to points in other groups. The user must specify the number of clusters (a dimensionless count, *k*) in which to partition the array. Specifically, a k medoids partitioning is such that each point in the data set is associated with the cluster that minimizes the sum of the pair-wise distances between the data points and their associated cluster centers (medoids). This approach is analogous to k means, but uses actual data points (the medoids) as the cluster exemplars instead of the means. Medoids in this context refer to the data point in each cluster that is most like all other data points, i.e., that data point whose average distance to all other data points in the cluster is smallest. Unlike k means, since pair-wise distances are minimized instead of variance, any arbitrary concept of "distance" may be used; this **Filter** allows for the selection of a variety of distance metrics. diff --git a/src/Plugins/SimplnxCore/docs/RandomizeFeatureIdsFilter.md b/src/Plugins/SimplnxCore/docs/RandomizeFeatureIdsFilter.md index c45c6a50d5..3bdf53f01d 100644 --- a/src/Plugins/SimplnxCore/docs/RandomizeFeatureIdsFilter.md +++ b/src/Plugins/SimplnxCore/docs/RandomizeFeatureIdsFilter.md @@ -6,7 +6,7 @@ Core (Filters) ## Description -***WARNING:** This filter can throw a pipeline terminating error (at runtime) if the number of tuples in the supplied Feature `Attribute Matrix` is less than the max value in the Feature Ids `DataArray`* +**WARNING:** *This filter can throw a pipeline terminating error (at runtime) if the number of tuples in the supplied Feature `Attribute Matrix` is less than the max value in the Feature Ids `DataArray`.* This filter will randomize a user selected **Feature Ids** array and update every container (`DataArray`, `NeighborList`, and `StringArray`) in the Feature **Attribute Matrix**. This does not generate random data but instead uses the existing values and swaps the positions of the values in the array. The intended use case is primarily for visualization, so feature data does not appear as a smooth gradient. diff --git a/wrapping/python/docs/Makefile.in b/wrapping/python/docs/Makefile.in index 98ee97508e..0c1067c227 100644 --- a/wrapping/python/docs/Makefile.in +++ b/wrapping/python/docs/Makefile.in @@ -3,7 +3,7 @@ # You can set these variables from the command line, and also # from the environment for the first two. -SPHINXOPTS ?= +SPHINXOPTS ?= -W --keep-going SPHINXBUILD ?= ${SPHINX_BUILD_EXECUTABLE} SOURCEDIR = ${SIMPLNX_SPHINX_DOCS_DIR}/source BUILDDIR = ${SIMPLNX_SPHINX_DOCS_DIR}/build @@ -17,4 +17,4 @@ help: # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile - @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) > ${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log 2>&1 + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) > "${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log" 2>&1 || { status=$$?; cat "${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log"; exit $$status; } diff --git a/wrapping/python/docs/generate_sphinx_docs.cpp b/wrapping/python/docs/generate_sphinx_docs.cpp index db90254737..8f9f605eb3 100644 --- a/wrapping/python/docs/generate_sphinx_docs.cpp +++ b/wrapping/python/docs/generate_sphinx_docs.cpp @@ -12,6 +12,7 @@ #include "plugin_source_dirs.hpp" +#include #include #include #include @@ -304,45 +305,137 @@ std::string GenerateUnderline(size_t length, char value) return result; } -std::string MarkdownToRstLink(const std::string& markDown) +namespace { - std::string rst = markDown; - - // Find the start of a Markdown link - size_t start = rst.find('['); - while(start != std::string::npos) +std::string EscapeRst(const std::string& text) +{ + std::string rst; + for(const char value : text) { - size_t const endText = rst.find(']', start); - if(endText == std::string::npos) + if(value == '\\' || value == '*' || value == '`' || value == '|' || value == '_') { - break; // No matching ']' + rst += '\\'; } + rst += value; + } + return rst; +} - size_t const startUrl = rst.find('(', endText); - if(startUrl == std::string::npos) +usize FindMarkdownSpanEnd(const std::string& text, usize start, const std::string& delimiter) +{ + const usize contentStart = start + delimiter.size(); + for(usize end = contentStart; end < text.size(); ++end) + { + if(delimiter != "`" && text[end] == '\\') { - break; // No matching '(' after ']' + ++end; + continue; } - size_t const endUrl = rst.find(')', startUrl); - if(endUrl == std::string::npos) + // A code span is opaque, including any asterisks inside it. + if(delimiter != "`" && text[end] == '`') { - break; // No matching ')' + const usize codeEnd = text.find('`', end + 1); + if(codeEnd != std::string::npos) + { + end = codeEnd; + continue; + } } - - // Extract link text and URL - const std::string linkText = rst.substr(start + 1, endText - start - 1); - const std::string url = rst.substr(startUrl + 1, endUrl - startUrl - 1); - - // Replace with reStructuredText format - const std::string rstLink = fmt::format("`{} <{}>`_", linkText, url); - rst.replace(start, endUrl - start + 1, rstLink); - - // Find the next link - start = rst.find('[', endUrl); + if(text.compare(end, delimiter.size(), delimiter) != 0) + { + continue; + } + const usize spanEnd = end + delimiter.size(); + if(delimiter != "`" && ((end > contentStart && text[end - 1] == '*') || (spanEnd < text.size() && text[spanEnd] == '*'))) + { + continue; + } + if(end > contentStart && std::isspace(static_cast(text[contentStart])) == 0 && std::isspace(static_cast(text[end - 1])) == 0) + { + return end; + } + return std::string::npos; } + return std::string::npos; +} +std::string MarkdownInlineToRst(const std::string& markDown, bool plainText = false) +{ + std::string rst; + for(usize index = 0; index < markDown.size();) + { + const char value = markDown[index]; + usize spanEnd = index; + std::string converted; + if(value == '\\' && index + 1 < markDown.size() && std::ispunct(static_cast(markDown[index + 1])) != 0) + { + rst += EscapeRst(markDown.substr(index + 1, 1)); + index += 2; + continue; + } + if(value == '[') + { + const usize textEnd = markDown.find(']', index + 1); + if(textEnd != std::string::npos && textEnd > index + 1 && markDown.compare(textEnd, 2, "](") == 0) + { + const usize urlEnd = markDown.find(')', textEnd + 2); + if(urlEnd != std::string::npos && urlEnd > textEnd + 2) + { + const std::string label = MarkdownInlineToRst(markDown.substr(index + 1, textEnd - index - 1), true); + const std::string url = EscapeRst(markDown.substr(textEnd + 2, urlEnd - textEnd - 2)); + converted = plainText ? label : "`" + label + " <" + url + ">`_"; + spanEnd = urlEnd + 1; + } + } + } + else if(value == '`' || value == '*') + { + usize delimiterSize = 1; + if(value == '*') + { + while(index + delimiterSize < markDown.size() && markDown[index + delimiterSize] == '*') + { + ++delimiterSize; + } + } + const std::string delimiter(delimiterSize, value); + const usize end = delimiterSize <= 3 ? FindMarkdownSpanEnd(markDown, index, delimiter) : std::string::npos; + if(end != std::string::npos) + { + const std::string content = markDown.substr(index + delimiterSize, end - index - delimiterSize); + // RST cannot nest inline markup. Keep the outer style and flatten inner spans. + const std::string rstDelimiter = value == '`' ? "``" : (delimiterSize >= 2 ? "**" : "*"); + const std::string rstContent = value == '`' ? (plainText ? EscapeRst(content) : content) : MarkdownInlineToRst(content, true); + converted = plainText ? rstContent : rstDelimiter + rstContent + rstDelimiter; + spanEnd = end + delimiterSize; + } + else + { + rst += EscapeRst(delimiter); + index += delimiterSize; + continue; + } + } + if(spanEnd > index) + { + if(!plainText && index > 0 && std::isalnum(static_cast(markDown[index - 1])) != 0) + { + rst += "\\ "; + } + rst += converted; + if(!plainText && spanEnd < markDown.size() && std::isalnum(static_cast(markDown[spanEnd])) != 0) + { + rst += "\\ "; + } + index = spanEnd; + continue; + } + rst += EscapeRst(markDown.substr(index, 1)); + ++index; + } return rst; } +} // namespace std::string CreateFilledString(const std::string& input_string, int max_width, char fill_value) { @@ -385,33 +478,38 @@ std::string MarkdownToRst(const std::string& markDown) rst = std::regex_replace(rst, std::regex("^## (.+)$"), "$1\n" + GenerateUnderline(40, '-')); rst = std::regex_replace(rst, std::regex("^### (.+)$"), "$1\n" + GenerateUnderline(40, '^')); - // Convert bold text - rst = std::regex_replace(rst, std::regex("\\*\\*(.+)\\*\\*"), "**$1**"); - - // Convert italic text - rst = std::regex_replace(rst, std::regex("\\*(.+)\\*"), "*$1*"); - - // Convert unordered lists - rst = std::regex_replace(rst, std::regex("^\\s*[-*+] (.+)$"), "* $1"); - - // Convert ordered lists - rst = std::regex_replace(rst, std::regex("^\\s*\\d+\\. (.+)$"), "#. $1"); - - // Convert List - // rst = std::regex_replace(rst, std::regex("\\[(.+)\\]\\((.+)\\)"), "`$1 <$2>`_"); - - // Convert links - // rst = std::regex_replace(rst, std::regex("\\[(.+)\\]\\((.+)\\)"), "`$1 <$2>`_"); - rst = MarkdownToRstLink(rst); - - // Convert images - // rst = std::regex_replace(rst, std::regex("!\\[(.*)\\]\\((.+?)\\)"), ".. image:: $2\n :alt: $1"); - - // Convert inline code - // rst = std::regex_replace(rst, std::regex("`(.+?)`"), "``$1``"); - - // Convert code blocks - rst = std::regex_replace(rst, std::regex("```\n?(.+?)\n?```"), "::\n\n $1"); + // Keep code block contents out of the inline scanner. + std::smatch codeMatch; + if(std::regex_search(rst, codeMatch, std::regex("```\n?(.+?)\n?```"))) + { + return MarkdownToRst(codeMatch.prefix().str()) + "::\n\n " + codeMatch[1].str() + MarkdownToRst(codeMatch.suffix().str()); + } + // Convert each line separately so list markers are not treated as inline markup. + std::istringstream lines(rst); + std::string line; + const std::regex unorderedListRegex("^\\s*[-*+] (.+)$"); + const std::regex orderedListRegex("^\\s*\\d+\\. (.+)$"); + rst.clear(); + while(std::getline(lines, line)) + { + std::smatch listMatch; + if(std::regex_match(line, listMatch, unorderedListRegex)) + { + rst += "* " + MarkdownInlineToRst(listMatch[1]); + } + else if(std::regex_match(line, listMatch, orderedListRegex)) + { + rst += "#. " + MarkdownInlineToRst(listMatch[1]); + } + else + { + rst += MarkdownInlineToRst(line); + } + if(!lines.eof()) + { + rst += '\n'; + } + } rst = std::regex_replace(rst, std::regex(" (.+?)\n"), "|$1\n"); return rst; @@ -746,7 +844,7 @@ void GeneratePythonRstFiles() rstStream << parameterPair.first; std::string helpText = anyParameter->helpText(); - helpText = StringUtilities::replace(helpText, "\n", " "); + helpText = EscapeRst(StringUtilities::replace(helpText, "\n", " ")); memberStream << " :param nx." << nx::core::StringUtilities::replace(s_ParameterMap[anyParameter->uuid()], "simplnx.", "") << " " << anyParameter->name() << ": " << helpText << "\n"; } rstStream << ")\n\n"; diff --git a/wrapping/python/docs/make.bat.in b/wrapping/python/docs/make.bat.in index 51a47187aa..77f273b26d 100644 --- a/wrapping/python/docs/make.bat.in +++ b/wrapping/python/docs/make.bat.in @@ -1,18 +1,21 @@ @ECHO OFF +setlocal -pushd %~dp0 +pushd "%~dp0" REM Command file for Sphinx documentation -if "%SPHINXBUILD%" == "" ( - set SPHINXBUILD=${SPHINX_BUILD_EXECUTABLE} +if not defined SPHINXBUILD ( + set "SPHINXBUILD=${SPHINX_BUILD_EXECUTABLE}" ) -set SOURCEDIR=${SIMPLNX_SPHINX_DOCS_DIR}/source -set BUILDDIR=${SIMPLNX_SPHINX_DOCS_DIR}/build +if not defined SPHINXOPTS set "SPHINXOPTS=-W --keep-going" +set "SOURCEDIR=${SIMPLNX_SPHINX_DOCS_DIR}/source" +set "BUILDDIR=${SIMPLNX_SPHINX_DOCS_DIR}/build" +set "SPHINXLOG=${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log" if "%1" == "" goto help -%SPHINXBUILD% >NUL 2>NUL +"%SPHINXBUILD%" >NUL 2>NUL if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx @@ -22,14 +25,19 @@ if errorlevel 9009 ( echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ - exit /b 1 + set "SPHINXSTATUS=1" + goto end ) -%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% > ${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log +"%SPHINXBUILD%" -M %1 "%SOURCEDIR%" "%BUILDDIR%" %SPHINXOPTS% %O% > "%SPHINXLOG%" 2>&1 +set "SPHINXSTATUS=%ERRORLEVEL%" +if not "%SPHINXSTATUS%" == "0" type "%SPHINXLOG%" goto end :help -%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +"%SPHINXBUILD%" -M help "%SOURCEDIR%" "%BUILDDIR%" %SPHINXOPTS% %O% +set "SPHINXSTATUS=%ERRORLEVEL%" :end popd +endlocal & exit /b %SPHINXSTATUS% diff --git a/wrapping/python/docs/source/Developer_API.rst b/wrapping/python/docs/source/Developer_API.rst index 5f58c505ff..4d6bc81fde 100644 --- a/wrapping/python/docs/source/Developer_API.rst +++ b/wrapping/python/docs/source/Developer_API.rst @@ -481,7 +481,7 @@ General Parameters **Enumeration** - .. py:enum:: CropGeometryParameter.TypeEnum + .. py:class:: CropGeometryParameter.TypeEnum - ``NoCropping`` – do not crop. - ``VoxelSubvolume`` – use the ``*_bound_voxels`` members. diff --git a/wrapping/python/docs/source/conf.py b/wrapping/python/docs/source/conf.py index b5dc74f1c5..337871b5ef 100644 --- a/wrapping/python/docs/source/conf.py +++ b/wrapping/python/docs/source/conf.py @@ -30,4 +30,3 @@ html_theme = 'sphinx_rtd_theme' #html_theme = 'sphinx_book_theme' -html_static_path = ['_static']