shell monitors gpu usage

The output of a nvidia-smi looks like this:

Tue Aug 9 23:05:08 2022+-----------------------------------------------------------------------------+| NVIDIA-SMI 440.44 Driver Version: 440.44 CUDA Version: 10.2 ||-------------------------------+----------------------+----------------------+| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. ||================================+=======================+========================|| 0 TITAN Xp Off | 00000000:01:00.0 Off | N/A || 37% 58C P2 75W / 250W | 8481MiB / 12195MiB | 2% Default |+--------------------------------------------+----------------------+----------------------++-----------------------------------------------------------------------------+| Processes: GPU Memory || GPU PID Type Process name Usage ||=================================================================================|| 0 2292 C python 8467MiB |+-----------------------------------------------------------------------------+

Where 8481MiB / 12195MiB shows the used video memory and the total amount of video memory.Now use the shell command to calculate the available video memory size:

nvidia-smi | \grep -E "[0-9]+MiB\s*/\s*[0-9]+MiB" | \awk '{print ($9" "$11)}' | \sed "s/\([0-9]\{1,\}\)MiB \([0-9]\{1,\}\)MiB/\1 \2/" | \awk '{print $2 - $1}'

Where:

• | pipeline, pass the result of the previous command to the next command; \ continue;
• grep -E "[0-9]+MiB\s*/\s*[0-9]+MiB" Filter out the line of output that contains usage, get:

  | 37% 58C P2 75W / 250W | 8481MiB / 12195MiB | 2% Default |

• awk '{print ($9" "$11)}' filters out the core data of used video memory and total video memory (here are the 9th and 11th columns of the row),Open the space, get:

  8481MiB 12195MiB

• sed "s/\([0-9]\{1,\}\)MiB \([0-9]\{1,\}\)MiB/\1 \2/"Remove the MiB suffix (brackets enclose the matching group, \1, \2 refer to the matched string), get:

   8481 12195

• awk '{print $2 - $1}' Calculate the difference between two columns and calculate the available video memory, get:

  3714

This is the case of a single card, if there are multiple cards, maybe | should be used with xarg?