A pingback is a way for one blog to tell another blog that it has made a post about that blog.
It's like this. Say Adam makes a post on his blog. Bob then sees that post, and makes a post on his blog as well, and includes a link in his post to Adam's post. Both blogs are pingback enabled, and so Bob's automatically sends a pingback to Adam's. What Adam's blog does with it is up to the blog itself, but in the case of WordPress, Adam's will make a comment with a link back to Bob's blog entry.
That's a pingback. A trackback is similar, but slightly different and more manual. Pingbacks are fully automatic.
In either case, what that looks like is not spam so much as it is that somebody commented on your blog entry and posted a link to you in the process. Thus that text is a bit of their post which is on their own blog.
Not everybody blogs in English. I get an amazing number of links to me from Japanese blogs.
Part of the comment will have a link back to their own blog. Take a look at it and see if it looks like it's really a blog or a splog (a fake blog which mainly has spam on it). If it's real, I'd leave it.
Now, pingbacks and trackbacks (mostly trackbacks) are abused by spammers a lot, however this is just the nature of the web. If you have spam problems, your best move is to install Akismet and Bad Behavior. Bad Behavior because it blocks obvious spammers instantly, and Akismet because it uses a centralized spam detection system, and so by using it, not only will it block virtually all spam, but it will contribute back to the system and thus help to block all blog spamming everywhere else too.