For people that know Ruby, if you haven't explored Ruby's CLI abilities, you definitely, definitely should[1][2]. When I was building my (free and open source) awk course[3] I fell in love with awk. When I later found out that Ruby has some of the same features, it changed my life:
Little underappreciated part of Ruby these days is that part of it grew out of Perl, and specifically supports similar kind of scripting for daily tasks. Pretty sure both thsoe specific switches and some of the sigils used in syntax came from this Perl legacy.
My mnemonic for the perl switches is "perl dash lane", i.e. `perl -lane`. And then I remember to replace "n" with "p" if I want to auto-print every line after munging it. I mainly use this as a replacement for sed/awk/grep because PCRE syntax is more familiar to me than whatever flavor(s) of regexes are used by those tools.
Yes thanks, I should have added that perl took it (and improved it) from awk, and ruby took it from perl. Most ruby people don't like to remember that ruby has a lot of perl-isms in it, but I think perl was a great language (and probably is, though I liked ruby so much better I never looked back).
I've been daily driving xonsh for a couple of years too. It _sounds_ like a terrible idea for reasons you can't quite put your finger on, but other than a minor annoyance every now and then, it's been great.
I especially like iterating over files as Path objects with the simple syntax
for f in p`.\*\.jpg`:
if f.isdir():
foo = $(echo @(f.absolute()))
Really it's just a souped-up Python interactive shell. When you give it bash commands, it forks to bash. When you give it python it evaluates like a REPL
No new syntax, unless you want to write scripts in xonsh, or mix bash and Python in the same command. There's rarely any reason to. 99/100 times I just write pure bash or pure python
I love the motivation for this tool. This looks very useful for when you just want to make something easier in an existing shell script, without fully migrating to Python. If you do want to migrate fully to Python, I encourage people to use sh[0] (disclaimer: author).
One concern that I have is that it could be so useful that now your script is filled with `| pz`, and is launching Python processes at every line . Now you have a tangled hybrid of Bash and Python.
Dealing with a lot of `| pz` statements could be pretty quickly solved with eg a `pz` interpreter that can be explicitly given multiple sequences of inputs.
That said, it would be amazing if `pz` could take in some input arguments and output the Unixy equivalent if it can find one.
Or pz could be a lightweight wrapper that speaks to a python server or starts a new one if there isn't yet. The server could end after X seconds of not getting new requests.
I don't know if I like this approach. Common "regular" Bash programs have a huge advantage - they are common. When I share my 'may accidentally conjure up a monster' one-liners with my team, I am sure they can read it, understand it, and run it. I lose that advantage with the use of a very nice, but obscure tool.
Bash involves tons and tons of arcania. It also doesn't support features that PL editors have like Intellisense. I sometimes feel like the only people who don't see a problem with the Unix experience are people who've used it day-in day-out for a decade, and don't need to grep man pages more than they'd like to. But by then, it's Stockholm Syndrome? I think this Github project might begin to pull out the chord. I know my Python better than I know my awk/sed/bash/xargs/find/make/bc...
Bash has one major advantage though: it’s core libraries and and commands almost never change which mean your scripts are most likely going to work forever. Python core library development is filled with asshats who like to deprecate naming conventions in later versions rather than just improving the existing libraries and keeping the method names the same. Python is great for a lot of reasons but if you know bash well, it is far more reliable in the long term.
> Python core library development is filled with asshats who like to deprecate naming conventions in later versions rather than just improving the existing libraries and keeping the method names the same.
I think this is a gross misrepresentation of both technologies, to the point that it goes well beyond a normal apples-to-oranges comparison.
A shell script runs well on any standard-compliant implementation, just like python3.4 scripts will run exactly like they always did when running on a python3.4 interpreter.
Python's libraries break when asshats "deprecate naming conventions" only when you somehow assume that, say, your python3.10 script would run on python2.7, or just like your shell script breaks when the program it calls happens to break it's cli.
Also, is versioning still a problem given the pervasiveness of package managers with support for version pinning?
I'm 44 years old. When I read your comment, I think "Great, the Bash script will continue to work the next time I need it (which might be after Elon lands humans on another planet)". But when my 22 year old colleague reads it, he likely thinks "Great, that script is full of incomprehensible syntax from the era of my last diaper change".
Thinking long term is a skill that is often realized only after one fails to make a previous long-term investment.
Most of the examples would be relatively simple Perl or AWK one-liners. The appeal of this tool seems mostly to be that you can use Python, which isn't one-line friendly out of the box.
Probably easier to stick to bash rather than learn a new set of commands from scratch. Also bash commands are universally available whereas this needs to be installed first which is not always that straightforward. Also python is slower at some of the examples than compiled executables.
“Universal” is too much of a stretch. You still get tripped by things like “that flag doesn’t exist in OSX sed” or “that option in git was only available in git > 2.8.14”
And that’s how you end up with bash scripts wrapped inside Docker images, that I have been seeing for the last couple of years.
> Probably easier to stick to bash rather than the overhead of learning new set of commands that I need to learn from scratch.
That's all fine and dandy until your shell script grows beyond a dozen lines of code or so. The moment that happens, shell scripts become barely readable and terribly hard to maintain, unlike alternatives such as python.
I have to say I have not found this to be true. Bash is an excellent glue language if what you want to do is fundamentally a shell like activity (eg composing together other more powerful primitives). I think the key (which is often forgotten) is just to remember to use the abstractions provided by the language (also shell-check, which is the most helpful linter I have ever encountered)
If you use functions to aid in abstraction, you can easily convert the function to a program in a more capable language as soon as it becomes too complicated.
In the end, I usually find that most of the things people replace good old unix tools with are better in one limited area, but miss a large part of what make the originals magic. Usually this is some combination of universal, discoverable, reduced dependencies, composable abstractions.
For people familiar with python, the first one is immediately understandable, and the second one is incomprehensible. Learning Perl may have been a better choice long-term, but this might be a good stop-gap fix for those who know Python already.
> Caveat: When accessed first time, the auto-import makes the row reprocessed. It may influence your global variables. Use verbose output to see if something has been auto-imported.
It sounds like this could cause some very subtle bugs. Maybe a strong and clear warning when a line is being reprocessed would help.
I wrote a similar tool a while back. My solution for this was to statically analyse the input to discover unused names. The cool part of doing transformations statically is that you're able to see the exact Python code you'd run without running it, e.g. `pyp --explain x[:5]`.
https://github.com/hauntsaninja/pyp
What I would like to see (and haven't found yet), is a bash or python shell that automatically does a "more" when a command exceeds some number of lines.
So many times I accidently fill up my scroll buffer with an unintended awk or grep mistake.
As someone who considers themselves ‘not a real programmer’ but uses Python, shell scripts, etc to ‘just get things done’, I really love simple utilities like this.
Maybe it’s not the ‘right’ way to get things done, but it’s quick and it works.
[1]: https://robm.me.uk/2013/11/ruby-enp/
[2]: https://benoithamelin.tumblr.com/ruby1line/
[3]: https://github.com/FreedomBen/awk-hack-the-planet