当前位置：网站首页>An open source scripting scheme for quickly tracking close contacts

An open source scripting scheme for quickly tracking close contacts

2022-04-22 23:09:00 【InfoQ】

demand

Under the new crown epidemic , The flow of close contacts to specific personnel is very important . Flow regulation works in a variety of ways , Like video , Telephone , Data analysis, etc . The data analysis here is naturally our IT The task of the worker . That is, it needs to be based on all kinds of behavior data , Go to find close contacts with specific people in time and space .

The sources of data are clearly diverse , May include the network , consumption , Access control and other data , At the same time, the demand of flow regulation is also changing from time to time , We need to abstract this scene , Form a unified model , In order to facilitate the subsequent automatic processing .

Model

What I want to dig is close contacts with specific people in time and space , So whatever the type of raw data , We should always focus on only three fields .

User ID —— He should be unique to every natural person

The time of behavior —— Through time , We can be right ” space-time ” Medium “ when ” It's positioned .

Where the behavior takes place —— For example, wireless AP name , The end of consumption ID etc. , The specific content is not important , As long as he can locate the space where the behavior occurs .

After determining these three elements , We need to further describe “ Spatiotemporal adjoint ” The concept of .“ Time ” Accompanying , We can understand it as a behavior record with a limited time range , and “ Space ” Accompanying , It can be regarded as a limited number of behavior records in a specific location .

So a flow regulation job , Whether it's automated or manual query , Generally follow the following process ：

Inquire about “ Transferred personnel ” Record your behavior , Lock three elements

According to the obtained behavior records , Further trace back to the front and back generated at the same position N A person with a record of behavior

If the behavior record exceeds the limited time , It ignores

Script

It is easy to base on the above models and algorithms , Realize automatic streaming script . We open source an implementation scheme , For your reference —— https://github.com/ECNU/contact_tracing

Running environment

python3

Dependent installation

pip3 install -r requirement.txt

To configure

Database and close connection parameters

modify

settings.py

The configuration file , Configure basic parameters

#  Database configuration 
database_type=&quot;sqlite&quot; # Three database types are supported ：mysql、oracle、sqlite
user = &quot;&quot; #  Special character escape 
password = &quot;&quot; #  Special character escape 
host = &quot;&quot; #  Special character escape 
port=&quot;&quot; # Database connection port 
database=&quot;&quot; #  Special character escape , When the connection  oracle  when , Equivalent to  oracle  Connected  service
db_path=&quot;./example.db&quot; #  Absolute path , When the database is sqlite It needs to be filled in when , and user、password、host、port、database No need to fill in 

#  Related to database tables 
table_name = &quot;example&quot; #  Table name 
user_id_field_name = &quot;USERID&quot; #  Represent user id Field name 
time_field_name = &quot;RECORDTIME&quot; #  The name of the field representing the time 
location_filed_names = [&quot;XIAOQU&quot;,&quot;TERMNAME&quot;] #  A field representing a location , It could be multiple , For example, there is a campus field plus a location field 

#  Output result field  ( Related to the field names defined in the database table , The default should be ： user id Field of 、 Field name of time 、 Location field ; The program automatically adds a user close to the user id Field ;)
result_filed_names = [&quot;USERID&quot;,&quot;RECORDTIME&quot;,&quot;XIAOQU&quot;,&quot;TERMNAME&quot;,&quot;TYPE&quot;]


#  Time range 
flow_tone_start_time = &quot;2022-03-01 00:00:00&quot;
flow_tone_end_time = &quot;2022-03-10 00:00:00&quot;

#  Close contact limits the number of people 
close_contact_people_num = 5

#  Close contact limits time  （ minute ）
close_contact_time = 30

Input parameters

stay

USERID.xlsx

Fill in the form with

userid

, Each row of a

perform

stay

src

Execute under directory

main.py

cd src/
python3 main.py

Output

stay
result
Under the table of contents , The file named after the running time is the streaming output

The line marked in yellow is the input consumption record of the flow dispatcher

Running logic

First step , Read the input user_id, One by one

Query a user_id Consumption records during flow adjustment time

Conduct downstream query according to each record , Query the consumption records in the same place, at the same time and within several minutes in the record , Take the number of people in a limited range .

Output all records to the specified table and highlight the cell of the input person

practice

In our open source repository , We provide an example of

sqlite

Data sets . Practical application , We can set the data sets that need to be checked for flow regulation , Build fields suitable for script execution in the way of view . If the database view on the production environment is inconvenient to adjust , You can also export the data later , The import script is applicable in other databases .

For data from different sources , It can be integrated through data synchronization , For unstructured data sources , You can extract the related field content , Structured integration . In short, the required data can be integrated and adjusted according to the format required by the model , This is also the daily work of data governance .

The script is based on

sqlalchemy

It's done

orm

programme , At present, it can support

mysql

、

oracle

、

sqlite

Three databases , It should be easy to further expand to

PG

and

MsSql

, Welcome to contribute

PR

This work will be carried out after the end of the epidemic , Data mining that can be transformed into social relationships . By analyzing the relationship between people “ Close contact “ And then tap the social relationship between people , Therefore, it still has good development potential . This may involve privacy and ethical issues , It is necessary to pay attention to relevant process compliance in the follow-up work .

Affirming

In the warehouse

example.db

Relevant data are randomly generated test data , For test verification only , There's no real meaning .

Only the above test data are used in the project development process , Not using any real personal information .

The developed script will be delivered to authorized staff for use , Developers have no right to obtain personal information data of production environment .

The project is self initiated by the community , This article is written by an individual , Does not represent an official position .

版权声明
本文为[InfoQ]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/04/202204222308320366.html