Add Shell method for running commands via the system shell

CONTRIBUTING.md

@@ -16,7 +16,7 @@ I _don't_ recommend just making a pull request for some new feature—it probabl
 
 ## Write a use case
 
-This is probably the most important thing to bear in mind. A great design principle for software libraries is to start with a real-world use case, and try to implement it using the feature you have in mind. _No issues or PRs will be accepted into `script` without an accompanying use case_. And I hold myself to that rule just as much as anybody else.
+This is probably the most important part of your issue. A great design principle for software libraries is to start with a real-world use case, and try to implement it using the feature you have in mind. _No issues or PRs will be accepted into `script` without an accompanying use case_. And I hold myself to that rule just as much as anybody else.
 
 What do I mean by "use case"? I mean a real problem that you or someone else actually has, that could be solved using the feature. For example, you might think it's a very cool idea to add a `Frobnicate()` method to `script`. Maybe it is, but what's it for? Where would this be used in the real world? Can you give an example of a problem that could be solved by a `script` program using `Frobnicate()`? If so, what would the program look like?
 
@@ -26,13 +26,23 @@ A concrete use case also provides a helpful example program that can be included
 
 The final reason is that it's tempting to over-elaborate a design and add all sorts of bells and whistles that nobody actually wants. Simple APIs are best. If you think of an enhancement, but it's not needed for your use case, leave it out. Things can always be enhanced later if necessary.
 
+# Stop
+
+Stop here until you've raised the issue with a use case, we've discussed it and agreed a solution design, and I've given you the go-ahead to write a PR.
+
+Once those things have happened, you can get cracking.
+
+# AI policy
+
+Use of AI is fine. Disclose it if you like.
+
 # Coding standards
 
 A library is easier to use, and easier for contributors to work on, if it has a consistent, unified style, approach, and layout. Here are a few hints on how to make a `script` PR that will be accepted right away.
 
 ## Tests
 
-It goes without saying, but I'll say it anyway, that you must provide comprehensive tests for your feature. Code coverage doesn't need to be 100% (that's a waste of time and effort), but it does need to be very good. The [awesome-go](https://github.com/avelino/awesome-go) collection (which `script` is part of) mandates at least 80% coverage, and I'd rather it were 90% or better.
+It goes without saying, but I'll say it anyway, that you should provide comprehensive tests for your feature. Code coverage doesn't need to be 100% (that's a waste of time and effort), but it does need to be very good. The [awesome-go](https://github.com/avelino/awesome-go) collection (which `script` is part of) mandates at least 80% coverage, and I'd rather it were 90% or better.
 
 Test data should go in the `testdata` directory. If you create a file of data for input to your method, name it `method_name.input.txt`. If you create a 'golden' file (of correct output, to compare with the output from your method) name it `method_name.golden.txt`. This will help keep things organised.
 
@@ -50,13 +60,9 @@ Add lots of test cases; they're cheap. Don't just test the obvious happy-path ca
 
 Remember people are using `script` to write mission-critical system administration programs where their data, their privacy, and even their business could be at stake. Now, of course it's up to them to make sure that their programs are safe and correct; library maintainers bear no responsibility for that. But we can at least ensure that the code is as reliable and trustworthy as we can make it.
 
-### Add your method to `doMethodsOnPipe` for stress testing
-
-One final point: a common source of errors in Go programs is methods being called on zero or nil values. All `script` pipe methods should handle this situation, as well as being called on a valid pipe that just happens to have no contents (such as a newly-created pipe).
-
-To ensure this, we call every possible method on (in turn) a nil pipe, a zero pipe, and an empty pipe, using the `doMethodsOnPipe` helper function. If you add a new method to `script`, add a call to your method to this helper function, and it will automatically be stress tested.
+### Concurrency safety
 
-Methods on a nil, zero, or empty pipe should not necessarily do nothing; that depends on the method semantics. For example, `WriteFile()` on an empty pipe creates the required file, writes nothing to it, and closes it. This is correct behaviour.
+Because pipes are concurrent (they may be running multiple commands or filter functions concurrently, for example), we use a mutex to access fields on the `Pipe` struct rather than reading or writing them directly. For example, don't read `p.Env` directly: use `p.environment()` instead.
 
 ## Dealing with errors
 
@@ -126,7 +132,6 @@ Here's a handy checklist for making sure your PR will be accepted as quickly as
 - [ ] Have you opened an issue to discuss the feature and agree its general design?
 - [ ] Do you have a use case and, ideally, an example program using the feature?
 - [ ] Do you have tests covering 90%+ of the feature code (and, of course passing)
-- [ ] Have you added your method to the `doMethodsOnPipe` stress tests?
 - [ ] Have you written complete and accurate doc comments?
 - [ ] Have you updated the README and its table of contents?
 - [ ] You rock. Thanks a lot.

README.md

@@ -10,8 +10,6 @@ import "github.com/bitfield/script"
 
 [![Magical gopher logo](img/magic.png)](https://bitfieldconsulting.com/subscribe)
 
-[Subscribe to learn Go with me!](https://bitfieldconsulting.com/subscribe)
-
 # What is `script`?
 
 `script` is a Go library for doing the kind of tasks that shell scripts are good at: reading files, executing subprocesses, counting lines, matching strings, and so on.
@@ -137,10 +135,10 @@ data, err := script.Do(req).JQ(".[0] | {message: .commit.message, name: .commit.
 We can also run external programs and get their output:
 
 ```go
-script.Exec("ping 127.0.0.1").Stdout()
+script.ExecCommand("ping", "127.0.0.1").Stdout()
 ```
 
-Note that `Exec` runs the command concurrently: it doesn't wait for the command to complete before returning any output. That's good, because this `ping` command will run forever (or until we get bored).
+Note that `ExecCommand` runs the command concurrently: it doesn't wait for the command to complete before returning any output. That's good, because this `ping` command will run forever (or until we get bored).
 
 Instead, when we read from the pipe using `Stdout`, we see each line of output as it's produced:
 
@@ -178,7 +176,7 @@ The `func` we supply to `Filter` takes just two parameters: a reader to read fro
 
 If our `func` returns some error, then, just as with the `Do` example, the pipe's error status is set, and subsequent stages become a no-op.
 
-Filters run concurrently, so the pipeline can start producing output before the input has been fully read, as it did in the `ping` example. In fact, most built-in pipe methods, including `Exec`, are implemented *using* `Filter`.
+Filters run concurrently, so the pipeline can start producing output before the input has been fully read, as it did in the `ping` example. In fact, most built-in pipe methods, including `ExecCommand`, are implemented *using* `Filter`.
 
 If we want to scan input line by line, we could do that with a `Filter` function that creates a `bufio.Scanner` on its input, but we don't need to:
 
@@ -237,7 +235,7 @@ If you're already familiar with shell scripting and the Unix toolset, here is a
 
 | Unix / shell       | `script` equivalent |
 | ------------------ | ------------------- |
-| (any program name) | [`Exec`](https://pkg.go.dev/github.com/bitfield/script#Exec) |
+| (any program name) | [`ExecCommand`](https://pkg.go.dev/github.com/bitfield/script#ExecCommand) |
 | `[ -f FILE ]`      | [`IfExists`](https://pkg.go.dev/github.com/bitfield/script#IfExists) |
 | `>`                | [`WriteFile`](https://pkg.go.dev/github.com/bitfield/script#Pipe.WriteFile) |
 | `>>`               | [`AppendFile`](https://pkg.go.dev/github.com/bitfield/script#Pipe.AppendFile) |
@@ -256,6 +254,7 @@ If you're already familiar with shell scripting and the Unix toolset, here is a
 | `jq`     | [`JQ`](https://pkg.go.dev/github.com/bitfield/script#Pipe.JQ) |
 | `ls`               | [`ListFiles`](https://pkg.go.dev/github.com/bitfield/script#ListFiles) |
 | `sed`              | [`Replace`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Replace) / [`ReplaceRegexp`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ReplaceRegexp) |
+| `sh`               | [`Shell`](https://pkg.go.dev/github.com/bitfield/script#Shell) |
 | `sha256sum`        | [`Hash`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Hash) / [`HashSums`](https://pkg.go.dev/github.com/bitfield/script#Pipe.HashSums) |
 | `tail`             | [`Last`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Last) |
 | `tee`              | [`Tee`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Tee) |
@@ -306,13 +305,14 @@ These are functions that create a pipe with a given contents:
 | [`Args`](https://pkg.go.dev/github.com/bitfield/script#Args) | command-line arguments |
 | [`Do`](https://pkg.go.dev/github.com/bitfield/script#Do) | HTTP response |
 | [`Echo`](https://pkg.go.dev/github.com/bitfield/script#Echo) | a string |
-| [`Exec`](https://pkg.go.dev/github.com/bitfield/script#Exec) | command output |
+| [`ExecCommand`](https://pkg.go.dev/github.com/bitfield/script#ExecCommand) | command output |
 | [`File`](https://pkg.go.dev/github.com/bitfield/script#File) | file contents |
 | [`FindFiles`](https://pkg.go.dev/github.com/bitfield/script#FindFiles) | recursive file listing |
 | [`Get`](https://pkg.go.dev/github.com/bitfield/script#Get) | HTTP response |
 | [`IfExists`](https://pkg.go.dev/github.com/bitfield/script#IfExists) | do something only if some file exists |
 | [`ListFiles`](https://pkg.go.dev/github.com/bitfield/script#ListFiles) | file listing (including wildcards) |
 | [`Post`](https://pkg.go.dev/github.com/bitfield/script#Post) | HTTP response |
+| [`Shell`](https://pkg.go.dev/github.com/bitfield/script#Shell) | command output, run via the system shell |
 | [`Slice`](https://pkg.go.dev/github.com/bitfield/script#Slice) | slice elements, one per line |
 | [`Stdin`](https://pkg.go.dev/github.com/bitfield/script#Stdin) | standard input |
 
@@ -344,7 +344,7 @@ Filters are methods on an existing pipe that also return a pipe, allowing you to
 | [`Do`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Do) | response to supplied HTTP request |
 | [`Echo`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Echo) | all input replaced by given string |
 | [`EncodeBase64`](https://pkg.go.dev/github.com/bitfield/script#Pipe.EncodeBase64) | input encoded to base64 |
-| [`Exec`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Exec) | filtered through external command |
+| [`ExecCommand`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ExecCommand) | filtered through external command |
 | [`ExecForEach`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ExecForEach) | execute given command template for each line of input |
 | [`Filter`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Filter) | user-supplied function filtering a reader to a writer |
 | [`FilterLine`](https://pkg.go.dev/github.com/bitfield/script#Pipe.FilterLine) | user-supplied function filtering each line to a string|
@@ -363,6 +363,7 @@ Filters are methods on an existing pipe that also return a pipe, allowing you to
 | [`RejectRegexp`](https://pkg.go.dev/github.com/bitfield/script#Pipe.RejectRegexp) | lines not matching given regexp |
 | [`Replace`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Replace) | matching text replaced with given string |
 | [`ReplaceRegexp`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ReplaceRegexp) | matching text replaced with given string |
+| [`Shell`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Shell) | filtered through the system shell |
 | [`Tee`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Tee) | input copied to supplied writers |
 
 Note that filters run concurrently, rather than producing nothing until each stage has fully read its input. This is convenient for executing long-running commands, for example. If you do need to wait for the pipeline to complete, call [`Wait`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Wait).
@@ -388,6 +389,8 @@ Sinks are methods that return some data from a pipe, ending the pipeline and ext
 
 | Version | New |
 | ----------- | ------- |
+| 0.25.0  | [`ExecCommand`](https://pkg.go.dev/github.com/bitfield/script#Pipe.ExecCommand) / [`Shell`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Shell) supersede `Exec` (thanks [Dhanalakshmi-D04](https://github.com/Dhanalakshmi-D04)) |
+|         | [`WithContext`](https://pkg.go.dev/github.com/bitfield/script#Pipe.WithContext) (thanks [billvamva](https://github.com/billvamva)) |
 | 0.24.1  | [`JQ`](https://pkg.go.dev/github.com/bitfield/script#Pipe.JQ) accepts JSONLines data |
 | 0.24.0  | [`Hash`](https://pkg.go.dev/github.com/bitfield/script#Pipe.Hash) |
 |         | [`HashSums`](https://pkg.go.dev/github.com/bitfield/script#Pipe.HashSums) |

go.mod

@@ -2,7 +2,7 @@ module github.com/bitfield/script
 
 go 1.25.0
 
-toolchain go1.26.4
+toolchain go1.26.6
 
 require (
 	github.com/google/go-cmp v0.5.9