Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Statistics

## Description

***NOTE:* This filter allows each **Cell** value to contribute to 0 or more bounding boxes (a 1-to-many architecture). This comes with the caveat that the *Standardize Data* option offered by [Compute Attribute Array Statistics](ComputeArrayStatisticsFilter.md) cannot be included as functionality in this filter.**
**NOTE:** This filter allows each **Cell** value to contribute to 0 or more bounding boxes (a 1-to-many architecture). This comes with the caveat that the *Standardize Data* option offered by [Compute Attribute Array Statistics](ComputeArrayStatisticsFilter.md) cannot be included as functionality in this filter.

This filter computes a variety of statistics for a given scalar array, restricted to the **Cells** that fall inside each supplied bounding box. Because boxes may overlap (or leave gaps), a single **Cell** may be counted toward several boxes, exactly one box, or none. The currently available statistics are array length, minimum, maximum, (arithmetic) mean, median, mode, standard deviation, summation, and number of unique values; any combination of these statistics may be computed. Any scalar array, of any primitive type, may be used as input. The type of each output array depends on the kind of statistic computed:

Expand Down
2 changes: 1 addition & 1 deletion src/Plugins/SimplnxCore/docs/ComputeKMeansFilter.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ DREAM3D Review (Clustering)

## Description

***Warning:* The randomness in this filter is not currently consistent between operating systems even if the same seed is used. Specifically between Unix and Windows. This does not affect the results, but the IDs will not correspond. For example if the Cluster Identifier at index one on Linux is 1 it could be 2 on Windows, the overarching clusters will be the same, but their IDs will be different.**
**Warning:** *The randomness in this filter is not currently consistent between operating systems even if the same seed is used. Specifically between Unix and Windows. This does not affect the results, but the IDs will not correspond. For example if the Cluster Identifier at index one on Linux is 1 it could be 2 on Windows, the overarching clusters will be the same, but their IDs will be different.*

This **Filter** applies the k means algorithm to an **Attribute Array**. K means is a *clustering algorithm* that assigns each point (each tuple) of the **Attribute Array** to a *cluster Id*. A **cluster** is a group of data points that are more similar to each other than to points in other groups. The user must specify the number of clusters (a dimensionless count, *k*) in which to partition the array.

Expand Down
2 changes: 1 addition & 1 deletion src/Plugins/SimplnxCore/docs/ComputeKMedoidsFilter.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ DREAM3D Review (Clustering)

## Description

***Warning:* The randomness in this filter is not currently consistent between operating systems even if the same seed is used. Specifically between Unix and Windows. This does not affect the results, but the IDs will not correspond. For example if the Cluster Identifier at index one on Linux is 1 it could be 2 on Windows, the overarching clusters will be the same, but their IDs will be different.**
**Warning:** *The randomness in this filter is not currently consistent between operating systems even if the same seed is used. Specifically between Unix and Windows. This does not affect the results, but the IDs will not correspond. For example if the Cluster Identifier at index one on Linux is 1 it could be 2 on Windows, the overarching clusters will be the same, but their IDs will be different.*

This **Filter** applies the k medoids algorithm to an **Attribute Array**. K medoids is a *clustering algorithm* that assigns each point (each tuple) of the **Attribute Array** to a *cluster Id*. A **cluster** is a group of data points that are more similar to each other than to points in other groups. The user must specify the number of clusters (a dimensionless count, *k*) in which to partition the array. Specifically, a k medoids partitioning is such that each point in the data set is associated with the cluster that minimizes the sum of the pair-wise distances between the data points and their associated cluster centers (medoids). This approach is analogous to k means, but uses actual data points (the medoids) as the cluster exemplars instead of the means. Medoids in this context refer to the data point in each cluster that is most like all other data points, i.e., that data point whose average distance to all other data points in the cluster is smallest. Unlike k means, since pair-wise distances are minimized instead of variance, any arbitrary concept of "distance" may be used; this **Filter** allows for the selection of a variety of distance metrics.

Expand Down
2 changes: 1 addition & 1 deletion src/Plugins/SimplnxCore/docs/RandomizeFeatureIdsFilter.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Core (Filters)

## Description

***WARNING:** This filter can throw a pipeline terminating error (at runtime) if the number of tuples in the supplied Feature `Attribute Matrix` is less than the max value in the Feature Ids `DataArray`*
**WARNING:** *This filter can throw a pipeline terminating error (at runtime) if the number of tuples in the supplied Feature `Attribute Matrix` is less than the max value in the Feature Ids `DataArray`.*

This filter will randomize a user selected **Feature Ids** array and update every container (`DataArray`, `NeighborList`, and `StringArray`) in the Feature **Attribute Matrix**. This does not generate random data but instead uses the existing values and swaps the positions of the values in the array. The intended use case is primarily for visualization, so feature data does not appear as a smooth gradient.

Expand Down
4 changes: 2 additions & 2 deletions wrapping/python/docs/Makefile.in
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXOPTS ?= -W --keep-going
SPHINXBUILD ?= ${SPHINX_BUILD_EXECUTABLE}
SOURCEDIR = ${SIMPLNX_SPHINX_DOCS_DIR}/source
BUILDDIR = ${SIMPLNX_SPHINX_DOCS_DIR}/build
Expand All @@ -17,4 +17,4 @@ help:
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) > ${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log 2>&1
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) > "${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log" 2>&1 || { status=$$?; cat "${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log"; exit $$status; }
206 changes: 152 additions & 54 deletions wrapping/python/docs/generate_sphinx_docs.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

#include "plugin_source_dirs.hpp"

#include <cctype>
#include <filesystem>
#include <fstream>
#include <iostream>
Expand Down Expand Up @@ -304,45 +305,137 @@ std::string GenerateUnderline(size_t length, char value)
return result;
}

std::string MarkdownToRstLink(const std::string& markDown)
namespace
{
std::string rst = markDown;

// Find the start of a Markdown link
size_t start = rst.find('[');
while(start != std::string::npos)
std::string EscapeRst(const std::string& text)
{
std::string rst;
for(const char value : text)
{
size_t const endText = rst.find(']', start);
if(endText == std::string::npos)
if(value == '\\' || value == '*' || value == '`' || value == '|' || value == '_')
{
break; // No matching ']'
rst += '\\';
}
rst += value;
}
return rst;
}

size_t const startUrl = rst.find('(', endText);
if(startUrl == std::string::npos)
usize FindMarkdownSpanEnd(const std::string& text, usize start, const std::string& delimiter)
{
const usize contentStart = start + delimiter.size();
for(usize end = contentStart; end < text.size(); ++end)
{
if(delimiter != "`" && text[end] == '\\')
{
break; // No matching '(' after ']'
++end;
continue;
}
size_t const endUrl = rst.find(')', startUrl);
if(endUrl == std::string::npos)
// A code span is opaque, including any asterisks inside it.
if(delimiter != "`" && text[end] == '`')
{
break; // No matching ')'
const usize codeEnd = text.find('`', end + 1);
if(codeEnd != std::string::npos)
{
end = codeEnd;
continue;
}
}

// Extract link text and URL
const std::string linkText = rst.substr(start + 1, endText - start - 1);
const std::string url = rst.substr(startUrl + 1, endUrl - startUrl - 1);

// Replace with reStructuredText format
const std::string rstLink = fmt::format("`{} <{}>`_", linkText, url);
rst.replace(start, endUrl - start + 1, rstLink);

// Find the next link
start = rst.find('[', endUrl);
if(text.compare(end, delimiter.size(), delimiter) != 0)
{
continue;
}
const usize spanEnd = end + delimiter.size();
if(delimiter != "`" && ((end > contentStart && text[end - 1] == '*') || (spanEnd < text.size() && text[spanEnd] == '*')))
{
continue;
}
if(end > contentStart && std::isspace(static_cast<unsigned char>(text[contentStart])) == 0 && std::isspace(static_cast<unsigned char>(text[end - 1])) == 0)
{
return end;
}
return std::string::npos;
}
return std::string::npos;
}

std::string MarkdownInlineToRst(const std::string& markDown, bool plainText = false)
{
std::string rst;
for(usize index = 0; index < markDown.size();)
{
const char value = markDown[index];
usize spanEnd = index;
std::string converted;
if(value == '\\' && index + 1 < markDown.size() && std::ispunct(static_cast<unsigned char>(markDown[index + 1])) != 0)
{
rst += EscapeRst(markDown.substr(index + 1, 1));
index += 2;
continue;
}
if(value == '[')
{
const usize textEnd = markDown.find(']', index + 1);
if(textEnd != std::string::npos && textEnd > index + 1 && markDown.compare(textEnd, 2, "](") == 0)
{
const usize urlEnd = markDown.find(')', textEnd + 2);
if(urlEnd != std::string::npos && urlEnd > textEnd + 2)
{
const std::string label = MarkdownInlineToRst(markDown.substr(index + 1, textEnd - index - 1), true);
const std::string url = EscapeRst(markDown.substr(textEnd + 2, urlEnd - textEnd - 2));
converted = plainText ? label : "`" + label + " <" + url + ">`_";
spanEnd = urlEnd + 1;
}
}
}
else if(value == '`' || value == '*')
{
usize delimiterSize = 1;
if(value == '*')
{
while(index + delimiterSize < markDown.size() && markDown[index + delimiterSize] == '*')
{
++delimiterSize;
}
}
const std::string delimiter(delimiterSize, value);
const usize end = delimiterSize <= 3 ? FindMarkdownSpanEnd(markDown, index, delimiter) : std::string::npos;
if(end != std::string::npos)
{
const std::string content = markDown.substr(index + delimiterSize, end - index - delimiterSize);
// RST cannot nest inline markup. Keep the outer style and flatten inner spans.
const std::string rstDelimiter = value == '`' ? "``" : (delimiterSize >= 2 ? "**" : "*");
const std::string rstContent = value == '`' ? (plainText ? EscapeRst(content) : content) : MarkdownInlineToRst(content, true);
converted = plainText ? rstContent : rstDelimiter + rstContent + rstDelimiter;
spanEnd = end + delimiterSize;
}
else
{
rst += EscapeRst(delimiter);
index += delimiterSize;
continue;
}
}
if(spanEnd > index)
{
if(!plainText && index > 0 && std::isalnum(static_cast<unsigned char>(markDown[index - 1])) != 0)
{
rst += "\\ ";
}
rst += converted;
if(!plainText && spanEnd < markDown.size() && std::isalnum(static_cast<unsigned char>(markDown[spanEnd])) != 0)
{
rst += "\\ ";
}
index = spanEnd;
continue;
}
rst += EscapeRst(markDown.substr(index, 1));
++index;
}
return rst;
}
} // namespace

std::string CreateFilledString(const std::string& input_string, int max_width, char fill_value)
{
Expand Down Expand Up @@ -385,33 +478,38 @@ std::string MarkdownToRst(const std::string& markDown)
rst = std::regex_replace(rst, std::regex("^## (.+)$"), "$1\n" + GenerateUnderline(40, '-'));
rst = std::regex_replace(rst, std::regex("^### (.+)$"), "$1\n" + GenerateUnderline(40, '^'));

// Convert bold text
rst = std::regex_replace(rst, std::regex("\\*\\*(.+)\\*\\*"), "**$1**");

// Convert italic text
rst = std::regex_replace(rst, std::regex("\\*(.+)\\*"), "*$1*");

// Convert unordered lists
rst = std::regex_replace(rst, std::regex("^\\s*[-*+] (.+)$"), "* $1");

// Convert ordered lists
rst = std::regex_replace(rst, std::regex("^\\s*\\d+\\. (.+)$"), "#. $1");

// Convert List
// rst = std::regex_replace(rst, std::regex("\\[(.+)\\]\\((.+)\\)"), "`$1 <$2>`_");

// Convert links
// rst = std::regex_replace(rst, std::regex("\\[(.+)\\]\\((.+)\\)"), "`$1 <$2>`_");
rst = MarkdownToRstLink(rst);

// Convert images
// rst = std::regex_replace(rst, std::regex("!\\[(.*)\\]\\((.+?)\\)"), ".. image:: $2\n :alt: $1");

// Convert inline code
// rst = std::regex_replace(rst, std::regex("`(.+?)`"), "``$1``");

// Convert code blocks
rst = std::regex_replace(rst, std::regex("```\n?(.+?)\n?```"), "::\n\n $1");
// Keep code block contents out of the inline scanner.
std::smatch codeMatch;
if(std::regex_search(rst, codeMatch, std::regex("```\n?(.+?)\n?```")))
{
return MarkdownToRst(codeMatch.prefix().str()) + "::\n\n " + codeMatch[1].str() + MarkdownToRst(codeMatch.suffix().str());
}
// Convert each line separately so list markers are not treated as inline markup.
std::istringstream lines(rst);
std::string line;
const std::regex unorderedListRegex("^\\s*[-*+] (.+)$");
const std::regex orderedListRegex("^\\s*\\d+\\. (.+)$");
rst.clear();
while(std::getline(lines, line))
{
std::smatch listMatch;
if(std::regex_match(line, listMatch, unorderedListRegex))
{
rst += "* " + MarkdownInlineToRst(listMatch[1]);
}
else if(std::regex_match(line, listMatch, orderedListRegex))
{
rst += "#. " + MarkdownInlineToRst(listMatch[1]);
}
else
{
rst += MarkdownInlineToRst(line);
}
if(!lines.eof())
{
rst += '\n';
}
}
rst = std::regex_replace(rst, std::regex(" (.+?)\n"), "|$1\n");

return rst;
Expand Down Expand Up @@ -746,7 +844,7 @@ void GeneratePythonRstFiles()

rstStream << parameterPair.first;
std::string helpText = anyParameter->helpText();
helpText = StringUtilities::replace(helpText, "\n", " ");
helpText = EscapeRst(StringUtilities::replace(helpText, "\n", " "));
memberStream << " :param nx." << nx::core::StringUtilities::replace(s_ParameterMap[anyParameter->uuid()], "simplnx.", "") << " " << anyParameter->name() << ": " << helpText << "\n";
}
rstStream << ")\n\n";
Expand Down
26 changes: 17 additions & 9 deletions wrapping/python/docs/make.bat.in
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
@ECHO OFF
setlocal

pushd %~dp0
pushd "%~dp0"

REM Command file for Sphinx documentation

if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=${SPHINX_BUILD_EXECUTABLE}
if not defined SPHINXBUILD (
set "SPHINXBUILD=${SPHINX_BUILD_EXECUTABLE}"
)
set SOURCEDIR=${SIMPLNX_SPHINX_DOCS_DIR}/source
set BUILDDIR=${SIMPLNX_SPHINX_DOCS_DIR}/build
if not defined SPHINXOPTS set "SPHINXOPTS=-W --keep-going"
set "SOURCEDIR=${SIMPLNX_SPHINX_DOCS_DIR}/source"
set "BUILDDIR=${SIMPLNX_SPHINX_DOCS_DIR}/build"
set "SPHINXLOG=${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log"

if "%1" == "" goto help

%SPHINXBUILD% >NUL 2>NUL
"%SPHINXBUILD%" >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
Expand All @@ -22,14 +25,19 @@ if errorlevel 9009 (
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
set "SPHINXSTATUS=1"
goto end
)

%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% > ${SIMPLNX_SPHINX_DOCS_DIR}/sphinx_build_output.log
"%SPHINXBUILD%" -M %1 "%SOURCEDIR%" "%BUILDDIR%" %SPHINXOPTS% %O% > "%SPHINXLOG%" 2>&1
set "SPHINXSTATUS=%ERRORLEVEL%"
if not "%SPHINXSTATUS%" == "0" type "%SPHINXLOG%"
goto end

:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
"%SPHINXBUILD%" -M help "%SOURCEDIR%" "%BUILDDIR%" %SPHINXOPTS% %O%
set "SPHINXSTATUS=%ERRORLEVEL%"

:end
popd
endlocal & exit /b %SPHINXSTATUS%
2 changes: 1 addition & 1 deletion wrapping/python/docs/source/Developer_API.rst
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@ General Parameters

**Enumeration**

.. py:enum:: CropGeometryParameter.TypeEnum
.. py:class:: CropGeometryParameter.TypeEnum

- ``NoCropping`` – do not crop.
- ``VoxelSubvolume`` – use the ``*_bound_voxels`` members.
Expand Down
1 change: 0 additions & 1 deletion wrapping/python/docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,3 @@

html_theme = 'sphinx_rtd_theme'
#html_theme = 'sphinx_book_theme'
html_static_path = ['_static']
Loading