如何解决操作复杂搜索的 Doctrine JSON 函数
大家好，希望你们一切顺利。
我在一个使用 MySQL 8 的 Symfony 4.4 项目中。该应用是 dockerized，并使用了一些其他服务，如 ELK 堆栈、Gophish、redis，但这些不考虑在内。
我正在尝试进行一些数据库操作来获取存储在 MySQL 中的一些事件。 这些事件包含一个类型（字符串）和一个负载（json）。为了进行 JSON 函数调用，我使用了 ScientaNL/DoctrineJsonFunctions/
为了给您提供上下文，我有一个向客户（合法）发送网络钓鱼电子邮件的平台，当用户执行诸如打开电子邮件、单击链接、提交数据等操作时会生成事件......那些事件由网络钓鱼服务器自动生成。
所以我的目标是根据最高风险创建用户执行操作的 csv 导出。
例如，用户打开了电子邮件，点击了链接。对于 csv，我只想显示“点击链接”事件
此外，每个事件的结构都具有自己的 Object 类，并且基于抽象层 Event 以实现一些基本方法，例如 log criticity 或 JsonSerializable。
然后将这些事件序列化为事件实体的负载：
<?php
namespace App\Entity;
use DateTime;
use DateTimeInterface;
use Doctrine\ORM\Mapping as ORM;
use Exception;
use JsonSerializable;
use Symfony\Component\Security\Core\User\UserInterface;
/**
* @ORM\Table("ht_event")
* @ORM\Entity(repositoryClass="App\Repository\EventRepository")
*/
class Event
{
/**
* @ORM\Id()
* @ORM\GeneratedValue()
* @ORM\Column(type="integer")
*/
private $id;
/**
* Event's type (See App\Enum\EventType).
*
* @ORM\Column(type="string",length=255)
*/
private $type;
/**
* Event's data.
*
* @var JsonSerializable|null
*
* @ORM\Column(type="json")
*/
private $data;
/**
* Event's creation date.
*
* @var DateTimeInterface
*
* @ORM\Column(
* name="created_at",* type="datetime"
* )
*/
protected $createdAt;
/**
* @var UserInterface|null
*
* @ORM\ManyToOne(targetEntity="App\Entity\User")
* @ORM\JoinColumn(nullable=true)
*/
private $user;
/**
* Event constructor.
*
* @throws Exception
*/
public function __construct()
{
$this->createdAt = new DateTime();
}
/**
* @return int|null
*/
public function getId(): ?int
{
return $this->id;
}
/**
* @return string|null
*/
public function getType(): ?string
{
return $this->type;
}
/**
* @param string $type
*
* @return $this
*/
public function setType(string $type): self
{
$this->type = $type;
return $this;
}
/**
* @return JsonSerializable|array|null
*/
public function getData()
{
return $this->data;
}
/**
* @param JsonSerializable|null $data
*
* @return $this
*/
public function setData(?JsonSerializable $data): self
{
$this->data = $data;
return $this;
}
/**
* @return UserInterface|null
*/
public function getUser(): ?UserInterface
{
return $this->user;
}
/**
* @param UserInterface|null $user
*
* @return $this
*/
public function setUser(?UserInterface $user): self
{
$this->user = $user;
return $this;
}
}
以下是 AbstractEvent 类的示例：
<?php
namespace App\EventStore;
use App\Enum\EventCriticityEnum;
use JsonSerializable;
use ReflectionClass;
use ReflectionException;
use Symfony\Component\Serializer\NameConverter\CamelCaseToSnakeCaseNameConverter;
class AbstractEvent implements JsonSerializable
{
/**
* @var string|null
*/
protected $sessionId;
/**
* Default criticity is EventCriticityEnum::CRITICITY_DEBUG.
*
* @var int
*/
protected $criticity = EventCriticityEnum::CRITICITY_DEBUG;
/**
* @return string|null
*/
public function getSessionId(): ?string
{
return $this->sessionId;
}
/**
* @param string|null $sessionId
*
* @return self
*/
public function setSessionId(?string $sessionId): self
{
$this->sessionId = $sessionId;
return $this;
}
/**
* @return int
*/
public function getCriticity(): int
{
return $this->criticity;
}
/**
* @param int $criticity
*
* @return self
*/
protected function setCriticity(int $criticity): self
{
$this->criticity = $criticity;
return $this;
}
/**
* @return array|mixed
*/
public function jsonSerialize()
{
try {
$reflect = new ReflectionClass($this);
} catch (ReflectionException $exception) {
return [];
}
$props = $reflect->getProperties(\ReflectionProperty::IS_PROTECTED);
$output = [];
foreach ($props as $prop) {
$getterName = sprintf('get%s',ucfirst($prop->getName()));
if (true === method_exists($this,$getterName)) {
// @Todo use it when data are normalized
// $snakeCaseconverter = new CamelCaseToSnakeCaseNameConverter();
// $jsonKey = $snakeCaseconverter->normalize($prop->getName());
// $output[$jsonKey] = $this->$getterName();
$output[$prop->getName()] = $this->$getterName();
}
}
return $output;
}
}
以下是特定事件的示例：
<?php
namespace App\EventStore\Phishing\Campaign;
use App\EventStore\AbstractEvent;
use DateTimeInterface;
use Ramsey\Uuid\UuidInterface;
/**
* Class PhishingCampaignTimelineEvent.
*/
class PhishingCampaignTimelineEvent extends AbstractEvent
{
/**
* @var int
*/
private $externalCampaignId;
/**
* @var int
*/
private $internalCampaignId;
/**
* @var UuidInterface
*/
private $companyUuid;
/**
* @var UuidInterface
*/
private $initiatiorUuid;
/**
* @var string
*/
private $email;
/**
* @var DateTimeInterface
*/
private $occurenceTime;
/**
* @var string
*/
private $message;
/**
* @var string
*/
private $details;
/**
* @return int
*/
public function getExternalCampaignId(): int
{
return $this->externalCampaignId;
}
/**
* @param int $externalCampaignId
*
* @return PhishingCampaignTimelineEvent
*/
public function setExternalCampaignId(int $externalCampaignId): self
{
$this->externalCampaignId = $externalCampaignId;
return $this;
}
/**
* @return int
*/
public function getInternalCampaignId(): int
{
return $this->internalCampaignId;
}
/**
* @param int $internalCampaignId
*
* @return PhishingCampaignTimelineEvent
*/
public function setInternalCampaignId(int $internalCampaignId): self
{
$this->internalCampaignId = $internalCampaignId;
return $this;
}
/**
* @return UuidInterface
*/
public function getCompanyUuid(): UuidInterface
{
return $this->companyUuid;
}
/**
* @param UuidInterface $companyUuid
*
* @return PhishingCampaignTimelineEvent
*/
public function setCompanyUuid(UuidInterface $companyUuid): self
{
$this->companyUuid = $companyUuid;
return $this;
}
/**
* @return UuidInterface
*/
public function getInitiatiorUuid(): UuidInterface
{
return $this->initiatiorUuid;
}
/**
* @param UuidInterface $initiatiorUuid
*
* @return PhishingCampaignTimelineEvent
*/
public function setInitiatiorUuid(UuidInterface $initiatiorUuid): self
{
$this->initiatiorUuid = $initiatiorUuid;
return $this;
}
/**
* @return string
*/
public function getEmail(): string
{
return $this->email;
}
/**
* @param string $email
*
* @return PhishingCampaignTimelineEvent
*/
public function setEmail(string $email): self
{
$this->email = $email;
return $this;
}
/**
* @return DateTimeInterface
*/
public function getOccurenceTime(): DateTimeInterface
{
return $this->occurenceTime;
}
/**
* @param DateTimeInterface $occurenceTime
*
* @return PhishingCampaignTimelineEvent
*/
public function setOccurenceTime(DateTimeInterface $occurenceTime): self
{
$this->occurenceTime = $occurenceTime;
return $this;
}
/**
* @return string
*/
public function getMessage(): string
{
return $this->message;
}
/**
* @param string $message
*
* @return PhishingCampaignTimelineEvent
*/
public function setMessage(string $message): self
{
$this->message = $message;
return $this;
}
/**
* @return string
*/
public function getDetails(): string
{
return $this->details;
}
/**
* @param string $details
*
* @return PhishingCampaignTimelineEvent
*/
public function setDetails(string $details): self
{
$this->details = $details;
return $this;
}
}
还有，如何使用这些对象来持久化事件：
/**
* @param UserInterface|null $user user to log for
* @param string $type event type
* @param JsonSerializable $data event data
*
* @throws Exception
*/
public function log(?UserInterface $user,string $type,JsonSerializable $data)
{
$event = new Event();
$event->setData($data);
$event->setUser($user);
$event->setType($type);
$this->em->persist($event);
$this->em->flush();
}
public function phishingCampaignTimelineEvent(/*ARGS HERE*/)
{
$event = new PhishingCampaignTimelineEvent();
$event->setSessionId()
->setCompanyUuid()
->setExternalCampaignId()
->setOccurenceTime()
->setInitiatiorUuid()
->setEmail()
->setMessage()
->setDetails();
$this->log(
$this->security->getUser(),EventDefinition::PHISHING_CAMPAIGN_TIMELINE_EVENT['name'],$event
);
}
这是我的存储库查询：
/**
* @param string $internalCampaignId
*
* @return Query
*/
public function findUserLatestPhishingTimelineEventByInternalCampaignId(string $internalCampaignId)
{
$qb = $this->_em->createQueryBuilder();
return $this->createQueryBuilder('e')
->andWhere("JSON_CONTAINS(e.data,:campaignId,'$.internal_campaign_id') = 1")
->andWhere($qb->expr()->in(
'e.id',$this->createQueryBuilder('event')
->select('event.id')
->andWhere("JSON_CONTAINS(event.data,JSON_EXTRACT(e.data,'$.initiator_uuid'),'$.initiator_uuid') = 1")
->andHaving("MAX(CAST(JSON_UNQUOTE(JSON_EXTRACT(e.data,'$.occurence_time')) as DATETIME))")
->getDQL()
))
->setParameter('campaignId',$internalCampaignId)
->getQuery();
}
最后是调用 repo 方法的服务函数：
/**
* This method gather the events typed 'EventDefinition::PHISHING_CAMPAIGN_TIMELINE_EVENT['name']' of a given phishing campaign.
* Once fetched a csv file is created to store those events and returned to the controller.
*
* @param ExportPhishingEventsFacade $facade
*
* @return false|resource
*/
public function exportEvents(ExportPhishingEventsFacade $facade)
{
/** @var CampaignHistory $campaign */
$campaign = $this->entityManagerInterface->getRepository(CampaignHistory::class)
->findOneBy(['providerCampaignId' => $facade->providerCampaignId]);
if (empty($campaign)) {
throw new ResourceNotFoundException(CampaignHistory::RESOURCE_NAME);
}
/** @var Query $eventList */
$eventList = $this->entityManagerInterface->getRepository(Event::class)
->findPhishingEventsByCampaign($campaign->getId());
$file = fopen('php://temp','w');
fputcsv($file,[
$this->translator->trans('phishing.export.csv.user-name.label.header',[],'phishing'),$this->translator->trans('phishing.export.csv.user-email.label.header',$this->translator->trans('phishing.export.csv.event.label.header',$this->translator->trans('phishing.export.csv.reported-mail.label.header',]);
$eventIterator = $eventList->iterate();
while (false !== ($line = $eventIterator->next())) {
if ('' === $initiatorUuid = $line[0]->getData()['initiator_uuid']) {
return false;
}
if (null === $user = $this->entityManagerInterface->getRepository(User::class)->findOneBy(['uuid' => $initiatorUuid])){
throw new ResourceNotFoundException(User::RESOURCE_NAME);
}
fputcsv($file,[
sprintf('%s %s',$user->getFirstName(),$user->getLastName()),$user->getEmail(),$line[0]->getData()['message'],]);
}
return $file;
}
此查询的目标是收集包含 CampaignID 的事件，使用子查询，迭代每个项目以按“initiator_uuid”过滤事件，然后使用最新日期进行记录。
问题是我不明白如何遍历每条记录以获取唯一用户的事件。首先我尝试在收集所有事件后进行排序，但它太消耗资源了。
如果有人得到提示或解决方案，我将不胜感激。
保重。
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。
