Synthetic Content Now Represents A Significant Threat!

Malicious Actors Almost Certainly Will Leverage Synthetic Content for Cyber and Foreign Influence Operations

Malicious actors almost certainly will leverage synthetic content for cyber and foreign influence operations in the next 12-18 months.

Foreign actors are currently using synthetic content in their influence campaigns, and the FBI anticipates it will be increasingly used by foreign and criminalCriminal A criminal is any person who through a decision or act engages in a crime. This can be complicated, as many people break laws unknowingly, however, in our context, it is a person who makes a decision to engage in unlawful acts or to place themselves with others who do this. A criminal always has the ability to decide not to break the law, or if they initially engage in crime to stop doing it, but instead continues. cyber actors for spearphishing and social engineeringSocial Engineering Social engineering is the psychological manipulation of people into performing actions or divulging confidential information. It is used as a type of confidence trick for the purpose of information gathering, fraud, or system access, it differs from a traditional "con" in that it is often one of many steps in a more complex fraud scheme. It has also been defined as "any act that influences a person to take any action that may or may not be in their best interests." in an evolution of cyber operational tradecraft.

Explaining Synthetic Content

The FBI defines synthetic content as the broad spectrum of generated or manipulated digital content, which includes images, video, audio, and text. While traditional techniques like Photoshop can be used to create synthetic content, this report highlights techniques based on artificial intelligenceArtificial intelligence Artificial intelligence (AI) is wide-ranging branch of computer science concerned with building smart machines capable of performing tasks that typically require human intelligence. Machine learning is a subset of Artificial Intelligence. (AIAI Artificial intelligence (AI) is wide-ranging branch of computer science concerned with building smart machines capable of performing tasks that typically require human intelligence. Machine learning is a subset of Artificial Intelligence.) or machine learningMachine learning Machine learning is a subset of artificial intelligence. Machine learning has the capacity to learn over time without being explicitly programmed. It can ingest a large amount of data and detect patterns and anomalies at scale. Supervised machine learning models are trained on a set of labels, while unsupervised machine learning does not require labeled data. (ML) technologies. These techniques are known popularly as deepfakes or GANs (generative adversarial networks). Generally, synthetic content is considered protected speech under the First Amendment. The FBI, however, may investigate malicious synthetic content which is attributed to foreign actors or is otherwise associated with criminal activities.

Recent and Anticipated Uses of Synthetic Content

Since late 2019, private sector researchers have identified multiple campaigns which have leveraged synthetic content in the form of ML-generated social media profile images.

Additionally, advances in AI- (artificial intelligence-based) and ML-based (machine learning-based) content generation and manipulation technologies likely could be used by malicious cyber actors to advance tradecraft and increase the impact of their activities. ML-generated profile images may help malicious actors spread their narratives, increasing the likelihood they will be more widely shared, making the message and messenger appear more authentic to consumers.

• Russian, Chinese, and Chinese-language actors are using synthetic profile images derived from GANs, according to multiple private-sector research reports. These profile images are associated with foreign influence campaigns, according to the same sources.
• Since 2017, unknown actors have created fictitious “journalists” who generated articles that were unwittingly published and amplified by a variety of online and print media outlets, according to press reports. These falsified personas often have a seemingly robust online presence, including the use of GANs profile images, however, basic fact checks can quickly reveal that the profiles are fraudulent.

Currently, individuals are more likely to encounter information online whose context has been altered by malicious actors versus fraudulent, synthesized content. This trend, however, will likely change as AL and ML technologies continue to advance

We anticipate malicious cyber actors will use these techniques broadly across their cyber operations—likely as an extension of existing spearphishing and social engineering campaigns, but with more severe and widespread impact due to the sophistication level of the synthetic media used.

• Malicious cyber actors may use synthetic content to create highly believable spearphishing messages or engage in sophisticated social engineering attacks, according to a late 2020 joint research report.

Synthetic content may also be used in a newly defined cyber-attack vector referred to as Business Identity Compromise (BIC). BIC will represent an evolution in Business Email Compromise (BEC) tradecraft by leveraging advanced techniques and new tools. Whereas BEC primarily includes the compromise of corporate email accounts to conduct fraudulent financial activities, BIC will involve the use of content generation and manipulation tools to develop synthetic corporate personas or to create a sophisticated emulation of an existing employee.

This emerging attack vector will likely have very significant financial and reputational impacts to victim businesses and organizations.

How to Identify and Mitigate Synthetic Content

• Visual indicators such as distortions, warping, or inconsistencies in images and video may be an indicator of synthetic images, particularly in social media profile avatars (profile images). For example, distinct, consistent eye spacing and placement across a wide sample of synthetic images provide one indicator of synthetic content.
• Similar visual inconsistencies are typically present in the synthetic video, often demonstrated by noticeable head and torso movements as well as syncing issues between face and lip movement, and any associated audio.
• Third-party research and forensic organizations, as well as some reputable cybersecurity companies, can aid in the identification and evaluation of suspected synthetic content.
• Finally, familiarity with media resiliency frameworks like the SIFT methodology can help mitigate the impact of cyber and influence operations.

The “SIFT” methodology encourages individuals to Stop, Investigate the source, Find trusted coverage, and Trace the original content when consuming information online.

Individuals and organizations can lower the risk of becoming victims to malicious actors using synthetic content by adopting good cyber hygiene and other security measures to include the following tips.

• Be aware of the potential for cyber or foreign influence activities using synthetic content. Be alert when consuming information online, particularly when topics are especially divisive or inflammatory;
• Seek multiple, independent sources of information;
• Do not assume an online persona or individual is legitimate based on the existence of video, photographs, or audio on their profile;
• Seek media literacy or media resiliency resources like SIFT, as well as training to harden individuals and corporate interests from the potential effects of influence campaigns;
• Use multi-factor authentication on all systems, especially on shared corporate social media accounts;
• Train users to identify and report attempts at social engineering and spearphishing which may compromise personal and corporate accounts;
• Establish and exercise communications continuity plans in the event social media accounts are compromised and used to spread synthetic content;
• Do not open attachments or click links within emails received from senders you do not recognize;
• Do not provide personal information, including usernames, passwords, birth dates, social security numbers, financial data, or other information in response to unsolicited inquiries;
• Be cautious when providing sensitive personal or corporate information electronically or over the phone, particularly if unsolicited or anomalous. Confirm, if possible, requests for sensitive information through secondary channels;
• Always verify the web address of legitimate websites and manually type them into your browser.

The FBI’s Protected Voices initiative provides additional tools and resources to companies, individuals, and political campaigns to protect against online foreign influence operations and cybersecurity threats.

Reporting Notice

The FBI encourages readers to report information concerning suspicious or criminal cyber activity to their local FBI field office. Additionally, corporations, individuals, or other entities who believe they are the target of foreign influence actors or other malign foreign entities are encouraged to contact their local FBI Field Office. Field office contacts can be identified at www.fbi.gov/contact-us/field-offices.

When available, each report submitted should include the date, time, location, type of activity, number of people, and type of equipment used for the activity, the name of the submitting company or organization, and a designated point of contact.
Administrative Note

TAGS: SCARS, Information About Scams, Anti-Scam, Scams, Scammers, Fraudsters, Cybercrime, Crybercriminals, Scam Victims, Online FraudFraud In law, fraud is intentional deception to secure unfair or unlawful gain (money or other assets), or to deprive a victim of a legal right. Fraud can violate civil law (e.g., a fraud victim may sue the fraud perpetrator to avoid the fraud or recover monetary compensation) or criminal law (e.g., a fraud perpetrator may be prosecuted and imprisoned by governmental authorities), or it may cause no loss of money, property, or legal right but still be an element of another civil or criminal wrong. The purpose of fraud may be monetary gain or other benefits, for example by obtaining a passport, travel document, or driver's license, or mortgage fraud, where the perpetrator may attempt to qualify for a mortgage by way of false statements. A fraud can also be a hoax, which is a distinct concept that involves deliberate deception without the intention of gain or of materially damaging or depriving a victim., Online Crime Is Real Crime, ScamScam A Scam is a confidence trick - a crime -  is an attempt to defraud a person or group after first gaining their trust through deception. Scams or confidence tricks exploit victims using their credulity, naïveté, compassion, vanity, irresponsibility, or greed and exploiting that. Researchers have defined confidence tricks as "a distinctive species of fraudulent conduct ... intending to further voluntary exchanges that are not mutually beneficial", as they "benefit con operators ('con men' - criminals) at the expense of their victims (the 'marks')". A scam is a crime even if no money was lost. Avoidance, Synthetic Content, Altered Images, Photoshopped Images, Fake Photos, FBI Warning