The short staff the son holds at a Chinese funeral is called the mourning staff (孝杖, xiao zhang). One explanation is that, long ago, it was a real walking stick.
When a parent died, the son kept vigil by the coffin, crying until he could not eat or sleep, and grew weaker by the day. Between mourning and arranging the funeral, many sons were worn out and needed a stick just to stay on their feet.
The Book of Rites (礼记) says the same: a son who loses a parent cries without end, and when his body grows ill and thin, he leans on a staff to support himself.
There are rules for the staff too. Bamboo is used for a father's funeral and wood for a mother's.
The Book of Rites specifies a bamboo staff for a father and a trimmed staff of paulownia wood for a mother. Why these two? One explanation from the Han dynasty text Bai Hu Tong is that the words sound like feelings: bamboo (竹) like “heart-wrenching” (蹙), and paulownia (桐) like “pain” (痛).
Passed down over generations, the walking stick became shorter and smaller, until it turned into the small stick the son holds today, a gesture of respect rather than a support.
Today some families no longer use it at all. At many modern farewell services, mourners in black wear only a hemp mourning band on the arm, without a mourning hat or staff. Each family follows its own practice.
Customs differ from place to place and between dialect groups. This is shared for reference only, with no need to be superstitious. How your family does it is best decided together with your elders.