Gems are only deducted when used. If you notice a lower balance, it indicates they were spent on one of the following in-game actions:
1. Restoring or Continuing a streak
Gems can be used to maintain your bingo streak, preventing it from resetting and allowing you to earn more rewards. 

2. Cannon blast
This option lets you fire a Cannon Blast, which adds extra daubs after the round ends. It gives you another chance to complete a bingo or secure extra rewards. 
3. Purchasing power-ups from the Store
Gems may also be spent in the Store to buy specific power-ups or bundles.

