Yann LeCun, an expert in computer vision and pattern recognition who was hired by Facebook (FB) in 2013, presented his research at a recent conference in Boston. He told New Scientist magazine that he wanted to see whether the same kinds of algorithms used for facial recognition could be tweaked to recognize people from other physical characteristics—their body type, the way they stand, etc.

“There are a lot of cues we use. People have characteristic aspects, even if you look at them from the back,” said LeCun, a former BellLabs researcher who helped develop the algorithm used by many U.S. banks to verify handwriting on checks. “For example, you can recognise Mark Zuckerberg very easily, because he always wears a gray T-shirt.”

The research took 40,000 public photos from the social network, some of which showed people with their faces fully visible to the camera and others with their faces partially or fully hidden. After running them through the recognition filter, LeCun said the system could determine a user’s identity with 83% accuracy. Using its existing algorithms, Facebook has said that it can recognize you with 98% accuracy—in fact, its software can identify you in one picture out of 800 million in less than 5 seconds.

Companies like Facebook are interested in facial recognition so that they can help users organize their photos, the way the social network wants to do with its recently launched Moments feature—which automatically sorts your pictures into different categories, and can detect when you and your friends upload photos of the same event. The description of the new service says:

“Moments groups the photos on your phone based on when they were taken and, using facial recognition technology, which friends are in them. You can then privately sync those photos quickly and easily with specific friends, and they can choose to sync their photos with you as well.”

Google is also focusing on similar features, which can recognize people, places and things in pictures and automatically organize or tag them for search. LeCun said that his recognition algorithm could be useful in helping people find out when someone else uploads a photo of them to Facebook, even if their face is not visible, which would help the privacy-conscious keep track of where their pictures are being published.

For many users, however, these kinds of features can cross a line where helpful becomes creepy. The same recognition algorithm that allows Facebook or Google to detect a picture of your child so you can share it easily could be used to identify people in all kinds of ways—many of them disturbing. For example, police or government authorities can track your location or behavior for their own purposes, or insurance companies could monitor your activities to see if your claim is justified.

[fortune-brightcove videoid=4278647741001]

Privacy advocates and industry groups are divided over how to approach this growing area. A meeting that was designed to find common ground between the two disintegrated recently after privacy groups said the industry representatives refused to agree that people have a right not to be identified in public places.

“What facial recognition allows is a world without anonymity,” Alvaro Bedoya of the Georgetown University Law Center told New Scientist. “You walk into a car dealership and the salesman knows your name and how much you make. That’s not a world I want to live in.” There are even churches using facial-recognition technology to identify those who attend regularly so they can ask them for donations.

Facebook’s pursuit of facial-recognition technology could also run into opposition from lawmakers in a number of states. The social network is already being sued by a man in Illinois who argues that the company’s auto-tagging feature breaches his rights under the state’s privacy laws. But as Fortune‘s Jeff Roberts explained recently, the legal aspects of facial recognition technology—and who is going to enforce any restrictions on companies like Facebook—is still very much an open question.