Tuesday, January 22, 2019

Challenges in Developing and Testing PHP Script for Automatic Resume Submission

In this post I would like to share my experience of creating and testing a php/MySQL script for posting candidate resume automatically to a lot of U.S Job sites.

i-e If a job seeker enters his Resume details in one place, the script will automatically create User Accounts in many U.S job sites for him, and it will automatically post his Resume details in all of these Job Sites.

Actually we made this script many years back. Though we are successful in creating the script, we found difficulty in marketing it. So, we dropped that project.

If you want me to develop any similar challenging scripts, you can read this post and contact me rajamanickam.a@gmail.com

This Kind of Script development is NOT an easy task. It involves lot of complexities. And, we haven’t started doing it properly. Initially we did similar script for one particular customer as a custom development, and later we enhanced it further to bring this Automatic Resume posting script.

Since we were a small Team, we just did Adhoc Testing only.

The important Steps involved in this kind of script development are,

– Automatic resume posting can be done in two different ways. One is, using javascript for auto filling the Forms in the job sites by accessing the DOM of the web page in the Client Side (i-e User’s browser). The other way is, posting the job seeker resume details to the Job Site using php cURL at the http protocol level. (i-e From Server where the Script is Hosted)

The second way (i-e php cURL) is involving some risks. Since the data for all job seekers are getting posted to job sites from same IP Address (i-e same server), the job sites may block that IP address if they thought that it is a DDoS attack.
But I came to know that almost all job sites are encouraging to automatically receive genuine Resume details of Job seekers to increase their customer base. That’s why, they are allowing continuous posting from particular IP. So, I decided to go with the php curl approach.

– First we created set of Function libraries for posting data in various situations (e.g GET method, POST method, with SSL, etc). Unit Testing of these functions is done by hardcoding the resume details in php files.

– Then, we did mapping of various fields of various job sites in mysql tables. (e.g The Input Resume will say experience as 3 years, but one job site will be having option for specifying experience  as “2-4 Years”, and another one will be having the option as “more than 2 years”. We need to map them manually onetime in mysql tables)

– Once after completing Function library creation and Job sites Fields mapping, we started creating php files for each job sites for getting the resume details and mapping details from db and post them to the corresponding job site. Live HTTP header firefox Add-on helped much to do this job. Some job sites will be having Captcha entry. The php script is developed to show the captcha image while doing resume posting.

– Then, created few php files for integrating posting of all job sites and for updating the status of account creation (account creation may fail if the user name already exists in particular job site) and resume posting process.

– Then, created user interface for allowing the Job Seekers to make payment, use discount codes, see reports, do Auto login into job sites, etc. And, created Admin Interface for managing users and resume posting.

I think I have given enough details about the background of this script. Now, let us start discussing about Testing this script.

The important challenge of Testing this Resume submission Script is, we can NOT have any separate Testing Environment. For testing our Time Sheet application, first we setup timesheet script in our local machine, and we will move the code to server only after completing the testing in local machine. We can do whatever we want to do testing in the local machine. But the same thing can not be done for testing the Resume submission script. Even if we setup the script in local machine the script need to post the details to the external job sites only.

So, we can not Test the script with junk data as it will spam the job sites.

The Next challenge for Testing the script is, the difficulty of choosing appropriate Test data. As I specified earlier, data Mapping plays important role in this script. So, if you decide to test the script with various test data to test the script including the quality of mapping, we will end with thousands of test data. So, it is NOT feasible to test the script completely by giving Test data. The only workaround is, we need to manually check the mapping in mysql table, by going thro’ a lot of mapping data. It is really difficult task and we will be missing to find some incorrect mappings.

Almost all job sites are updating their user interface frequently. Some job sites are getting closed and some of them are acquired by huge job sites. Some jobsites change their CMS completely. So, the script needs to be updated frequently.

As part of the regression testing, we needed to find some way for testing the entire script whenever we update the script. (either the coding or mapping), as just doing adHoc Testing is NOT enough.

So, I framed one approach for testing the script with real data. Instead of testing the script with junk data, I decided to allow the Job seekers to use the Resume Posting Service at very cheap price, in return they need to inform us if they find any issue in the job postings. I created some Special Discount Codes for the Job Seekers who are interested in joining this plan.

I wanted to make sure that this approach is not doing any harm to the Job seekers due to the mistakes in the data mapping or script execution.  So, during this  trial/test phase, I asked our Team to manually check  the quality of resume postings by logining into  each and every job site once the script completed the automatic posting.  It is a valid approach as we can easily update any data if our Team find any issue.

But we found difficulty in finding  people interested to try this plan. So, I came up with another plan. I decided to sell few copies of this entire script at very low price to the people who are showing willingness to cooperate with us to increase the quality of the script and to maintain the quality of the script at long run even if the job sites are getting changed frequently.
I could sell the script to significant number of People. But this approach didn’t work out as most of the script buyers didn’t spend time for reporting the issues. It seems they bought the script to get details about how we developed this script so that they can use the same concept for various other parts of their businesses.

So, finally I had come up with an Investment plan. I created a Business Plan Document for this project for getting Investment from  other Investors. But I couldn't find suitable Investor and I stopped this project entirely as I was engaged in developing and promoting our Motivational Quotes Website TheQuotes.Net by adding a lot features, like Calendar Generator, Gallery, Facebook Cover App, Auto Quotes Tweet, Auto FB Posts, etc.

Right now, I don't have plans to restart this project as almost all of the Job Sites got changed entirely. 

If you want me to work as a freelancer to develop any similar challenging scripts, you can read this post and contact me rajamanickam.a@gmail.com

No comments:

Search This Blog