php删除文本文件中重复行的方法

本文实例讲述了PHP删除文本文件中重复行的方法。分享给大家供大家参考。具体分析如下：

这个PHP函数用来删除文件中的重复行，还可以指定是否忽略大小写，和指定换行符

rush:PHP;"> /** * RemoveDuplicatedLines * This function removes all duplicated lines of the given text file. * * @param string * @param bool * @return string */ function RemoveDuplicatedLines($Filepath,$IgnoreCase=false,$NewLine="\n"){ if (!file_exists($Filepath)){ $ErrorMsg = 'RemoveDuplicatedLines error: '; $ErrorMsg .= 'The given file ' . $Filepath . ' does not exist!'; die($ErrorMsg); } $Content = file_get_contents($Filepath); $Content = RemoveDuplicatedLinesByString($Content,$IgnoreCase,$NewLine); // Is the file writeable? if (!is_writeable($Filepath)){ $ErrorMsg = 'RemoveDuplicatedLines error: '; $ErrorMsg .= 'The given file ' . $Filepath . ' is not writeable!'; die($ErrorMsg); } // Write the new file $FileResource = fopen($Filepath,'w+'); fwrite($FileResource,$Content); fclose($FileResource); }

/**

• RemoveDuplicatedLinesByString
• This function removes all duplicated lines of the given string.
• @param string
• @param bool
• @return string
  */
  function RemoveDuplicatedLinesByString($Lines,$NewLine="\n"){
  if (is_array($Lines))
  $Lines = implode($NewLine,$Lines);
  $Lines = explode($NewLine,$Lines);
  $LineArray = array();
  $Duplicates = 0;
  // Go trough all lines of the given file
  for ($Line=0; $Line < count($Lines); $Line++){
  // Trim whitespace for the current line
  $CurrentLine = trim($Lines[$Line]);
  // Skip empty lines
  if ($CurrentLine == '')
  continue;
  // Use the line contents as array key
  $LineKey = $CurrentLine;
  if ($IgnoreCase)
  $LineKey = strtolower($LineKey);
  // Check if the array key already exists,// if not add it otherwise increase the counter
  if (!isset($LineArray[$LineKey]))
  $LineArray[$LineKey] = $CurrentLine;
  else
  $Duplicates++;
  }
  // Sort the array
  asort($LineArray);
  // Return how many lines got removed
  return implode($NewLine,array_values($LineArray));
  }

使用范例：

rush:PHP;"> // Example 1 // Removes all duplicated lines of the file definied in the first parameter. $RemovedLinesCount = RemoveDuplicatedLines('test.txt'); print "Removed $RemovedLinesCount duplicate lines from the test.txt file."; // Example 2 (Ignore case) // Same as above,just ignores the line case. RemoveDuplicatedLines('test.txt',true); // Example 3 (Custom new line character) // By using the 3rd parameter you can define which character // should be used as new line indicator. In this case // the example file looks like 'foo;bar;foo;foo' and will // be replaced with 'foo;bar' RemoveDuplicatedLines('test.txt',false,';');

希望本文所述对大家的PHP程序设计有所帮助。