• Hi,
    ive tried posting this on SimplePie bugs list, and SimplePie mailing list, with no help!

    if someone could help me!

    im processing feeds in WP using SimplePie, and keeping a note of all feeds im processing in a custom table with get_id(true); this is a mysql db with varchar(255);

    $token = (string)$item->get_id(true);

    but some feed items, are being processed 2 or sometimes 3 times, thereby causing duplicates.

    following is the code, the function “alreadyParsed()”, is not working correctly as it should, if someone could help.

    function doFeed()
    {
    	$url			= "http://news.google.com/news?pz=1&cf=all&ned=au&hl=en&output=rss";
    	$rss			= fetch_feed($url);
    
    	$items = 10;
    	$parsed_count = 0;
    
    	foreach ( $rss->get_items(0, $items) as $item ) {
    
    		$link 	= esc_url(strip_tags($item->get_permalink()));
    		$title	= esc_attr(strip_tags($item->get_title()));
    		if ( empty($title) ) $title = __('Untitled');
    
    		$date 	= $item->get_date('Y-m-d');
    		$token 	= (string)$item->get_id(true);
    
    		// as soon as you hit a dup, bail out //
    		if(alreadyParsed($date, $token) !== false) return array($parsed_count, sprintf(__("Hit a Dup! :%s."), $title));
    
    		// new feed, process it //
    		$process_id = doItem($item);
    
    		if($process_id > 0)
    		{
    			$sql = "INSERT INTO <code>&quot;. WP_INDEX_TABLE .&quot;</code> SET ";
    				$sql .= " <code>process_id</code> 		= '". $process_id			. "', ";
    				$sql .= " <code>process_token</code>	= '". $token			. "', ";
    				$sql .= " <code>process_date</code>	= '". $date 			. "'";
    
    			$result1 = $wpdb->query($sql);
    			$parsed_count++;
    			unset($sql, $result1);
    		}
    	}
     	unset($sql,$url);
    	return array($parsed_count, __('Feeds processed'));
    }
    
    function alreadyParsed(&$date, &$token)
    {
    	global $wpdb;
    	$sql 	= "SELECT count(*) as count FROM <code>&quot;. WP_INDEX_TABLE .&quot;</code> WHERE
    			<code>process_token</code> = '" . $token		."' AND
    			<code>process_date</code>  = '" . $date		."'";
    
    	$res	= $wpdb->get_row($sql);
    
    	if((int)$res->count > 0)
    		return true;
    	else
    		return false;
    }

    mailing list : here

    If anyone can help!

    M.

Viewing 2 replies - 1 through 2 (of 2 total)
Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘SimplePie; using get_id() is giving me duplicates’ is closed to new replies.