Allegiance on Github

[BackTrak]

After a few tries, I was unceremoniously given the finger from GitHub while trying to import Trac tickets.


Failed to convert a ticket #24: Array
(
[message] => You have triggered an abuse detection mechanism and have been temporarily blocked from content creation. Please retry your request again later.
[documentation_url] => https://developer.github.com/v3#abuse-rate-limits
)

Ticket #37 wouldn't convert either, will need to mess with it some more. But, it did import some stuff!

https://github.com/BackTrak/TracTest/issues

[KGJV]

I never tried to import trac to github. Do you have a backup of the trac db somewhere so I can do some tests?

[Wasp]

DB/SVN via PKK

TRAC

[KGJV]

i did some test with a modified version of tratihubis : https://github.com/ahelsing/tratihubis

I've managed to import 65 issues : https://github.com/kgersen/AllegianceTest/issues before the "anti-abuse system" of github kicked me.

tratihubis does some throttling to keep github anti-abuse happy but not enough it seems. But thanks to the "--skipExisting" option we can restart where it failed.

So I've relaunched it and will let it run. There is no labels yet.

I'll post here the method and the code changes to tratihubis when complete.

[KGJV]

ok here the method:
step 1. convert trac db to csv files


- install sqlite3 (apt-get install sqlite3)
- create and cd into an empty working directory
- create a trac2cvs.sql file and put this in it:

Code: Select all

-- All Trac tickets to convert.
.mode csv
.output tickets.csv
select id,    type,    owner,    reporter,    milestone,    status,    resolution,    summary,
    description,   time / 1000000 as time,    changetime  / 1000000 as changetime,    component, priority, keywords, cc 
from    ticket order    by id;

.output comments.csv
select    ticket,  time  / 1000000 as time, author,    newvalue 
from    ticket_change where    field = 'comment'    and newvalue <> '' order    by ticket, time;

.output attachments.csv
select    id,     filename,     time  / 1000000 as time,    author from    attachment order    by id asc;

- execute the script:

Code: Select all

sqlite3 /path/to/trac.db <trac2cvs.sql

this will generate 3 files in the current directory: tickets.csv comments.csv attachments.csv

step 2. install tratihubis and prepare the github project

(assume you're still in the working directory from step 1).
- install python (2.7.x) if not already installed
- install PyGithub with "sudo pip install PyGithub"
- create a test project on github (imported issues & milestones cannot be deleted so better test 1st)
- get a token on github (doc: https://help.github.com/articles/creati ... -line-use/ , full control repo rights are enough)
- create a tratihubis.cfg file and put this in it:

Code: Select all

[tratihubis]
token = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
user = kgersen
repo = AllegianceTest
tickets = ../tickets.csv
comments =../comments.csv
labels = type=defect: bug, type=enhancement: enhancement, resolution=wontfix: wontfix, type=task: question, resolution=duplicate: duplicate, resolution=invalid: invalid
convert_text = true
trac_url = https://trac.alleg.net/

change token, user and repo to yours.

- get my modified tratihubis:

Code: Select all

git clone https://github.com/kgersen/tratihubis.git

step 3. test then export for real


test the import script (this will pretend to do the export ):

Code: Select all

cd tratihubis
python tratihubis.py ../tratihubis.cfg

check for errors , should end with something like this "Finished pretend creating 357 issues from 357 tickets"

if you want to map users and export attachments, read the tratihubis.py file for informations.
to launch the export for real:

Code: Select all

python tratihubis.py --really --skipExisting ../tratihubis.cfg

this might take quite some time and might stop because of github rate limiting. In that case just wait for your reset (see https://developer.github.com/v3/#rate-limiting ) then restart the command. But the best way is to contact github support and ask them to whitelist you.

Once finished (or while the export is ongoing) you can check the issues on github web site. If all ok. then edit tratihubis.cfg and set repo to the real final repo and do step 3 again.

[BackTrak]

Hi KG, I don't have time to check it out now, I'll try to get at it this weekend. It doesn't look like it preserved the original Trac ticket numbers so that they line up with the Github issue numbers. If we can get those to line up, it would be perfect. That is something that trac2github did. Unfortunately, trac2github couldn't convert certain issues out of the box. So, I'll check this one out as well!

Thanks!

[KGJV]

QUOTE (BackTrak @ Mar 23 2016, 04:24 AM) Hi KG, I don't have time to check it out now, I'll try to get at it this weekend. It doesn't look like it preserved the original Trac ticket numbers so that they line up with the Github issue numbers. If we can get those to line up, it would be perfect. That is something that trac2github did. Unfortunately, trac2github couldn't convert certain issues out of the box. So, I'll check this one out as well!

Thanks!

We dont have control on github issue numbers but they're created sequentially starting from 1.

if you want a perfect match between trac & github, you'll need to make sure that in the tickets.csv file, the ticket id (first field) goes from 1 to the last without any missing number. if some are missing, just create a dummy/placeholder line for them (something ez to find in github later). but if the final repo already has issues it's game over, the import won't start from #1 (a solution is to clone (not fork) the repo and start with a fresh clone without existing issues, import the trac tickets then import the old github issues).

What's the point of keeping trac ticket numbers anyway ? each imported github issue has a comment with original trac number and the issue/comment issue 'links' (#number) are mapped to the correct values by tratihubis (see "Converting Trac Wiki Markup to Github Markdown" in tratihubis.py)

I haven't looked at trac2github because php aint really my thing ... may be it can do the job too.

[KGJV]

If you struggle too much with github, you might want to look at GitLab.com. may be they have a better svn+trac import system.
I'm not very familiar with it so I can't help much but it could be more suitable than github. worth a look at least.

[facembani223]

I never tried to import trac to github. Do you have a backup of the trac db

[BackTrak]

Hi facembani,

Wasp posted it above: http://www.freeallegiance.org/pkk/backup/

Let us know if you have any success!