‘Well it’s only passing mv a list of–’ yeah yeah yeah, I know, and that’s why I’m calling bullshit. It should be massively harder to execute filenames. Even if 1970s decisions make that the eternal hideous default: the lack of any idiot-proof standard workaround is incomprehensible.
StackOverflow’s full of competing one-liners and people pointing out how each one is considered harmful. The least-skeezy options use exec. That sentence should make anyone recoil in horror.
This is not a filename problem. This is a tool problem. If a single printable character is going to silently expand into a list of names, then for god’s sake, having it put each name in quotes should be fucking trivial.
This is not standard behavior. Got links to StackOverflow?
This works just fine in bash and zsh:
encoded_funky_name=IUAjJCVeJiooKVtde30nIjo7IGxzOyAmYW1wOyA8Plx8YH5gJChscykke1BBVEh9ISF8fCYmLmpwZw== # decodes to: !@#$%^&*()[]{}'":; ls; & <>\|`~`$(ls)${PATH}!!||&&.jpg mkdir temp cd temp mkdir temp2 touch "$(echo "${encoded_funky_name}" | base64 -d)" mv ./*.jpg temp2 ls -lR mv temp2/*.jpg . ls -lRNote that it is recommended to use “./” before a raw * in globs to avoid filenames beginning with “-” being interpreted by the command as special arguments (this is not necessary in the above example, but it is a good habit). See https://dwheeler.com/essays/filenames-in-shell.html#globbing for more info. e.g.:
touch ./-laR # Bad ls command: ls * # Good ls command: ls ./*having it put each name in quotes should be fucking trivial
This is effectively what happens, except that it’s actually more robust than that, since it also accounts for names with quotes and other special characters. Glob expansion does NOT output file names into the shell; it passes them as arguments to the command with no further shell processing. Arguments can contain any special characters.
Not sure why you’re seeing this behavior. Are you doing something with these file names before passing them to
mv? Storing them in variables? Saving them to files?It’s literally just
mv *.jpg /mnt/Example/Pictures.Multiple StackOverflow answers take this as read.
mv ./*.jpg /mnt/Example/Picturesworks fine, though, so thank you for the practical solution. I’m still disgusted there’s any filename that gets treated as… not a filename. That broken stair might be older than I am.Ahhh. That’s a confusing thread, with a couple confounding factors.
The OP in that thread is attempting to a batch rename, which is out of scope of the
mvcommand.If you are NOT trying to change the name of the file, I recommend always making the last argument of
mvyour destination directory, rather than the destination file. This is strictly mandatory if you are moving multiple files.OP in that SO thread used the filename as the destination path — a filename which did not exist, and therefore could not be resolved by the wildcard. And if it had existed, the move would have failed. Because of that,
mv /data/*/Sample_*/logs/*_Data_time.err /data/*/Sample_*/logs/*_Data_time_orig.erris not a valid command.OP cannot accomplish what they want with a single
mvcommand. They would need to use a loop or something likefind.I agree that the lack of universal consistency in how commands handle arguments is unfortunate. There’s no way around that without reinventing the entire OS. Many commands let you pass
--as an argument to signify that any subsequent arguments should not be interpreted as options even if they begin with-. But that’s entirely up to the implementation of each program; it’s not a shell feature.Future OS designers: consider typed arguments. Abandon all legacy cruft.
There’s no way around that without reinventing the entire OS.
Disagree. If wildcard expansion is not a feature of the tool, it shouldn’t be up to the tool to shape how each expanded input is passed. Some hideous escape syntax ought to force
*.jpgto appear as"-a.jpg"instead of-a.jpg, once it reachesmv.Instead
*is acting as a cheeky littlelsalternative, minus any degree of formatting control, and piping actuallsinput intomvis similarly fraught with complications.Some hideous escape syntax ought to force *.jpg to appear as “-a.jpg” instead of -a.jpg, once it reaches mv
The issue here is that there’s no difference here between
-a.jpgand"-a.jpg".Go ahead and try these commands. They are 100% equivalent:
ls -laR ls "-laR" ls '-laR'The reason these are equivalent is because quoting (and expansion, and globbing) is processed in the shell. The shell then passes the evaluated string as an argument to the tool. All of those evaluate to the same string, so they look exactly the same to the
lscommand.What is the solution if there’s a file name that matches something the tool thinks is an argument? What would the correct behavior be?
The only solution is to include some way to communicate to the tool what each argument is. In practice, you could do
ls -- *and it would work even if the first filename was “-laR”. However, not all tools use the--convention. This is really core to the design of the OS, in that command arguments have no type.Jesus. That’s beyond Javascript levels of “helpfully” reinterpreting strings. That’s borderline Excel behavior.
What is the point of strings allowing every character besides
\0if they’re gonna eat quotation marks?All I can tell you is that, in my opinion, it’s ridiculous and terrible that old-school terminals haven’t been replaced yet with something more user-friendly and self-explanatory, at least in the same-machine user space. But given that they are what they are, some basic understanding of what shells do is required in order to use them, and you don’t have that understanding (I don’t fault you for this).
The key point here is that programs/commands always receive an array of string arguments, and it’s the shell’s job to translate your command line into that. Quoting (like in
-m="my message"), variable expansion (like$HOME) and various other things are processed by the shell and not the program, and the expectation is that the user knows this. So quotes are never visible to programs, and the upside is that programs never need to process such quotes - making these features universal without having various random bugs in each program’s implementation.This is exactly correct. This is why I say it would require a whole new OS with no regard for compatibility with current systems.
As long as arguments are typeless and transmitted in a single array, there is no universal way to make programs distinguish between an option and file name that are identical. Full stop. No way. This has nothing to do with quoting schemes.
There are conventions to work around this (like
--) but that is at the command level, NOT the shell level. It is not universal.I’ve been using command-line programs for twenty-five years. “Basic” is not the issue, here. This is obscene edge-case behavior for what honestly should be a Hello World level example.
Thoroughly explicable causes do not make the outcome any less of a problem.
Note that it is recommended to use “./” before a raw * in globs to avoid filenames beginning with “-” being interpreted by the command as special arguments
Jackpot! I think! Based on OP’s reply to me, this is almost definitely the real problem:
mv *.jpg /mnt/Example/PicturesWhere it will then die with an error like
mv: invalid option -- '1'depending on the contents of the current directory.
What is wrong with mv *.jpg?
I’m also a bit lost. From the final sentence “having it put each name in quotes should be fucking trivial.” OP seems to complain that filenames with spaces expand into multiple arguments. I can’t recreate this with this command in bash:
$ printf %s\\n *.jpgThis command prints each argument in its own line. Replace
*.jpgwith whatever to prove it to yourself. Filenames with spaces show in full in a single line for me.It expands
*.jpginto a list of names.Which you’d think would mean
mv "thing.jpg" "other thing.jpg", but no, it doesmv thing.jpg other thing.jpg- and then chokes, becauseotheris not a valid option or filename. Possibly after writing the contents ofother thing.jpgonto the destinationthing.jpg.If any filenames contain characters like
;or&, you better hope the next word doesn’t look like a command.Which shell are you using? This does not happen in Bash, Zsh, or any modern shell I’m familiar with.
What shell are you using? Is this exclusive to
mv(and maybe other builtins)?Edit: also why would you ever want to move files like this, what is wrong with you
Edit: also why would you ever want to move files like this, what is wrong with you
… like what?
The fuck do you mean, why would I want to move files in a terminal? Because I’m doing shit in a terminal! What, do you expect people to Super+E, find the directory they’re in, and drag-and-drop? I’m using tools that only exist in the command line.
I’m using whatever GNOME Terminal came with Linux Mint 20 by default. StackOverflow answers sound like any POSIX terminal will do this. I assume it’s standard behavior.
I’m using whatever GNOME Terminal came with Linux Mint 20 by default.
Should be bash. Type
helpit should tell you.StackOverflow answers sound like any POSIX terminal will do this. I assume it’s standard behavior.
Link?
I mean with * where you have no control of the order of the files and you never explicitly say which files are being moved. “yeah, just rename files into other files, I don’t fucking care lol” is what your command does
I have two questions:
- Is this some recent news that you expect us to have heard of? Your post demands context that is not easily understood (but it’s possible to understand through multiple reads)
- What shell are you using?
I’m using bash (
GNU bash, version 5.1.16(1)-release) and don’t observe this behavior:$ ls 'first file.txt' 'second file.txt' $ cat 'first file.txt' this is originally named "first file.txt" $ cat 'second file.txt' this is originally named "second file.txt" $ mv *.txt $ ls 'second file.txt' $ cat 'second file.txt' this is originally named "first file.txt"It’s possible that whatever shell you’re using has a bug, but it’s definitely not universal. It’s also possible that this bug only affects builtins like
mvand not non-builtin commands.I expected people have tried to move files… using the move command.
I did not expect I’d have to specify that there’s a destination, in an explanation of what
*.jpgdoes, not an explanation of whatmvdoes.For clarity:
GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)mv *.jpg /mnt/Example/PicturesWhere it will then die with an error like
mv: invalid option -- '1'depending on the contents of the current directory.Every search about this has led back to this StackOverflow page sooner or later. All answers address wildcard expansion.
Can you run
printf %s\\n *.jpgin that same directory and share the result? I’m really intrigued. If there’s private information in there, I’d be satisfied with just a handful of lines showing the issue with private info replaced with ****sEdit: also run
alias mvto check whether you have an alias messing things upbash: alias: mv: not foundprintf %s\\n *.jpgis just a list of filenames on newlines. There’s 75,000 of them. You’ll have to take my word for it.Fortunately, GenderNeutralBro’s aside about using
./*instead of*works as expected. Could have been files beginning with_-. None begin with-, exactly.
When I run into situations like this, I use the commands that work to write out a script. Eg, in your case the wildcard isn’t working with the mv command, so do something like this:
ls -1 *.jpg | awk ‘{print “mv "”$1"" /mnt/example/Pictures"}’ > /tmp/movefiles.sh
Then check the movefiles.sh and make sure it has all of the commands and files properly stated, make that executable, and then run that.
ls | grep | mvwould work, except the StackOverflow discussion also highlights how parsing ls can have the same issues.I am moving thousands of files at once. If I have to check each one, it’s still wrong.
The pragmatic answer turns out to be
./*instead of*.I saw that answer and was just offering another option. I am sure xargs might work, but you would need to test as you need a destination passed on each line. Back to my way, I have used it for a lot more than just the move command. I think I used it to do a chmod once where I wanted to check and make sure before I committed to actually running the command(s). You could also use find and the -exec option, which I think was also mentioned here.
Edit: also, you wouldn’t need to check each one, just the first few and last few to make sure the syntax is correct. Maybe do a wc -l to make sure it’s got the right number of entries and then let it run.
